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1.1. MULTI-SENSORY PROCESSING 


The brain receives signals about the environment 
from multiple senses simultaneously, and faces the 
challenge of integrating that information. This is the 
"binding" problem - "inferring which signals come from a 
common source and thus should be integrated. At the same 
time, it must also estimate the reliability of each 
Signal to determine how much influence it should have on 
the perceptual judgement. Effective decision making thus 
requires the brain to grapple with the uncertainty 
associated with various sensory signals as well as their 
causal relationship" (Fetsch and Noppeney 2023 pl) ?. 

Stange et al (2023) explained: "The predictive 
coding framework proposes that top-down predictions and 
bottom-up sensory input are compared at sensory 
processing stages; if predictions do not match the actual 
sensory input, this framework assumes that an error 
Signal is generated which travels downstream to update 
representations about the world. In line with these 
assumptions, both brain imaging and electrophysiological 
studies have revealed enhanced neural responses to 
unexpected compared to expected sensory events, which 
were interpreted as error signals" (pl). 


' Zeng et al (2023) described the example where "a prey urgently needs to confirm whether a cloud of 
spots behind the tussock are from a predator in order to decide whether to stay or flee. This task can be 
even more difficult when visual information is vague, for example, on a foggy day, or during the 
evening time. There are two possible ways to facilitate making more correct and faster choices. One is 
to observe whether these spots move coherently. The other is to collect evidence from other sources, 
such as footprints or smell, at the same time. This process of accumulating sensory evidence across 
time and modalities into categorical decisions to select the following course of action is defined as 
multi-sensory decision making (MSDM)" (p1). 
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Research on the brain's decision making processes in 
perception can be divided into three main themes (Fetsch 
and Noppeney 2023): 


1) The dynamic nature of the decision process - The 
sensory information "usually arrives not as a single 
snapshot but a temporal sequence or stream of 
information, and therefore we must decide not only what 
but also when to commit to a decision" (Fetsch and 


Noppeney 2023 p2). 


ii) The role of top-down factors - The decision 
making process is influenced by top-down factors like 
learning, expectations, and goals and intentions. The 
alternative is bottom-up processing which uses the 
sensory information alone. 


iii) Dealing with uncertainty - "Sensory information 
is imperfect, owing to limitations in our sensory 
apparatus, and to noise in both the physical events being 
transduced and the neural representation thereof. For 
this reason, perceptual judgements are productively 
studied within a framework of decision making under 
uncertainty" (Fetsch and Noppeney 2023 p2). 


Multi-sensory integration (MST) (table 1.1) involves 
many areas of the brain (both cortical and sub-cortical). 
Choi et al (2023) noted different types of MSI including 
speech MSI (the integration of sound of speech and vision 
of mouth movement), spatial MSI (information from 
different spatial locations), and temporal MSI (eg: audio 
and visual information at different times). 

The knowledge about the brain and MSI comes from 
three main sources - studies of the anatomy, measurement 
of electrical activity (electrophysiology), and 
neuroimaging experiments. The first brain area to be 
studied in detail was the superior colliculus (SC), which 
is a midbrain structure, and visual perception was 
investigated with single cell recording in anaesthetised 
cats (Meredith and Stein 1983) *. Subsequently, the SC was 
found to be involved in MSI (ie: auditory and visual 


? The early work by Stein et al (1975) found that "neurons in deep and middle, but not superficial 
layers of SC, respond to multi-modal (visual, auditory and somatic) stimuli. Importantly, the multi- 
sensory responses are much stronger than when only one modality is stimulated, often leading to a 
super-additive pattern (i.e. 1 + 1 > 2)" (Zeng et al 2023 pp1-2). Such early experiments, however, were 
performed on anaesthetised animals pr awake but passive ones. More recent studies measure neural 
responses in awake animals performing typical behaviours (eg: using a visuo-vestibular virtual-reality 
system). "In a typical experiment, awake monkeys are required to report whether their experienced 
heading direction is leftward or rightward compared with dead ahead by making a saccadic eye 
movement toward choice targets at the end of each trial" (Zeng et al 2023 p2). 
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inputs from a variety of brain areas. When neurons in the 
SC were destroyed (as in experiments with cats, say), the 
animals could not accurately orientate towards stimuli 
(Choi et al 2023). 

Research has shown the role of areas of the 
different cortices, the thalamus, and the amygdala in MSI 
(Choi et al 2023). 


Multi-sensory processing - "a host of brain functions that deal 
with multiple sensory modalities' inputs and states — for a 

variety of different purposes, including integration, 
suppression...). 


e Multi-sensory integration - "combining multiple sensory 
measurements from different modalities into a unified estimate 
of the stimulus". 


e Multi-sensory suppression - "the neuronal or behavioural 
responses to a stimulus are reduced by the presence of another 
stimulus from a different modality". 


e Multi-sensory interactions (or dependencies)- "when the 
response or measurement from one modality depends on the state 
(or input) of another". 


e Multi-sensory causal inference - "probabilistic inference 
regarding the origin(s) of the experienced stimuli". 


(Source: Zaidel and Salomon 2023 table 1) 


Table 1.1 - Key terms. 


Recent research is challenging long-held views about 
perception. Pennartz et al (2023) outlined one of them - 
"the canonical notion that the visual cortex is uniquely 
positioned to process only visual information, acting as 
a relay station to transmit this to higher order cortical 
and thalamic areas for decision making and other 
executive functions..." (pl). Work with rodents suggests 
that other sense information is also processed in the 
visual cortex and helps with visual perception, and that 
visual perception overall involves a wider network of 
brain areas (the "extended visual system"; Pennartz et al 
2023). 

Taking the first of the two challenges, neurons in 
the primary visual cortex respond to input from other 
senses (eg: auditory-evoked). In other words, there is 
multi-sensory processing in this area of the brain. But 
Pennartz et al (2023) made this point: "one may consider 
the alternative stance that the visual cortex is no 
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longer strictly 'visual'’ given all other inputs, and is 
therefore better characterised together with other 
sensory cortical domains — as 'multi-sensory cortex' 
[Ghazanfar and Schroeder 2006]. However, as visual 
cortical lesions cause blindness but not deafness (and 
vice versa for auditory cortical lesions...), the 
specific causal importance of visual cortex for visual 
perception needs to be accommodated as well" (p8). 

The "extended visual system" includes the visual 
cortex (Pennartz et al (2023) preferred to call the 
"photically driven cortex") as well as neurons in the 
prefrontal cortex, posterior parietal cortex, auditory 
cortex, and motor and somato-sensory areas of the cortex. 
The "cortical regions causally linked to non-visual 
sensory modalities can be considered to form satellite 
nodes contributing to the extended visual system..." 
(Pennartz et al 2023 p10). Zhang et al (2016) have mapped 
a wide-ranging visual network in the mouse brain. 

A key concept behind these ideas is "predictive 
processing", “holding that the perceiving brain forges 
best-guess representations (inferences) of the causes of 
sensory inputs. Thus, the brain is not tasked with 
exactly 'copying' features from the external world into 
perceptual systems, but does a much more constructive 
job: building an internal, representational model of what 
is likely causing the inputs relayed to the brain from 
the sense organs" (Pennartz et al 2023 p2) (appendix 1A). 


1.2. DYNAMIC NATURE OF THE DECISION PROCESS 


Flexibility in MSI is key as inputs vary in time and 
space across the different senses (modalities). A number 
of influences occur here, including (Choi et al 2023): 


1) Age - Age-dependent changes in the brain and MSI 
have been mapped in electrophysiological recordings of 
cat brains, for example (Wallace and Stein 1997). While 
in human infants, three stages of the development of MSI 
have been reported - "immature" (less than one month old) 
(respond to stimuli based on intensity), "broadly tuned" 
(5-8 months old) (increased attention to relevant 
stimuli), and "normally tuned" (12 months old) (eg: 
ability to perceive and respond to speech above other 
stimuli). Fine tuning of the MSI continues in childhood 
and into adulthood (Choi et al 2023). 


ii) Experience/learning - Feedback from previous 
experiences with particular stimuli influence MSI as 
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shown in neuroimaging experiments. For example, human 
participants are shown objects with sounds (congruent or 
incongruent) while the brain activity is measured. This 
is known as the "audio-visual object association task" 
(eg: Naumer et al 2009). 


1ii) Internal state - eg: stress. For example, mice 
had to navigate a T-maze in a situation of "audio-visual 
conflict" (ie: auditory and visual stimuli have opposite 
meanings) while stressed or not. When stressed, auditory 
stimuli were more important compared to visual stimuli as 
more important when not stressed (Choi et al 2023). 


iv) Deficits in MST in neurodevelopmental disorders 
- eg: autism. In the "temporal order judgment task", 
where visual and auditory stimuli are randomly presented 
and the task is to say which came first, children with 
autism are less accurate than healthy controls (Choi et 
al 2023). 


"To navigate and guide adaptive behaviour ina 
dynamic environment, animals must accurately estimate 
their own motion relative to the external world. This is 
a fundamentally multi-sensory process involving 
integration of visual, vestibular and kinaesthetic 
inputs" (Jerjian et al 2023 pl). This is "self-motion 
perception". Time is important here as the inf 


formation 
coming to the senses is constantly changing as we move. 
Jerjian et al (2023) added another key variable - 
certainty (ie: "the degree of confidence in a multi- 
sensory decision"; pl) °* *. 

Jerjian et al (2023) used the example of rock- 
climbing (at an indoor centre): "A successful, fast, 
climb to the top is facilitated by estimating an optimal 
route from an initial vantage point (or several) on the 
mat. During each movement across or up the wall, multiple 
sensory inputs are available to the brain to guide a 
successful climb: vestibular signals arising from motion 


* Zaidel and Salomon (2023) commented that "internal sensory information conveyed from 
proprioception (eg: where are my eyes looking?), vestibular (eg: how is my head positioned?) and 
interoception (eg: the sound is external and not my heartbeat) is seamlessly and implicitly taken into 
account to allow the explicit sensory experience of the world around us. All our sensory experiences 
are in actuality based on both internal and external multisensory information" (p1). 

“ Zaidel and Salomon (2023) emphasised that "sensory systems are not passive receivers. Rather, they 
are part of sensorimotor loops. Not only do the movements we make influence sensations, humans and 
animals actively move to sense the environment (active sensing). We move our eyes to probe the visual 
scene and move our hands to generate tactile input. This means that sensory signals can only be 
interpreted within a sensorimotor context, in addition to the mixing of signals from multiple senses. 
Thus, the observer must fundamentally determine whether each sensory observation is from an external 
(versus internal, self-generated) source to even be considered for integration" (p3). 
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of the head through space; visual signals from motion of 
the scene across the retina; proprioceptive and tactile 
Signals indicating the position and motion of the limbs 
and the quality of a hand- or foothold. Small or slippery 
holds may render tactile information unreliable. Visual 
input could be ambiguous or uncertain, for example if one 
is climbing on an overhang or with reduced ambient light 
levels. Depending on the frequency and amplitude of head 
motion, vestibular inputs may be unreliable or fail to 
disambiguate translation from tilt. Thus, to estimate 
their ongoing motion with respect to the goal and select 
actions accordingly, the optimal climber will use 
information from all available sources, at each moment 
instinctively leaning more heavily on the more reliable 
ones" (pl). Fast decision-making is the tim lement 
here, while "a climber's confidence that they have an 
accurate self-motion judgment is also critical" (Jerjian 
et al 2023 pl 

A number of cortical areas are known to be involved 
in human self-motion perception in this sort of activity, 
including the dorsal medial superior temporal area, and 
the ventral intra-parietal area (Jerjian et al 2023). 


~~ 
. 


The combination of visual and auditory information 
has been studied via "eye movement-related eardrum 
oscillations" (EMREOs) °. It was found that information 
about eye position and movements produced activity in the 
"predominantly auditory brain regions such as auditory 
cortex and the inferior colliculus during responses to 
sound stimuli" (Lovich et al 2023 p2). This led 
researchers to reason that "information about eye 
movements could be conveyed via the descending pathways 
to the motor actuators within the ear, such as the middle 
ear muscles and the outer hair cells, and that, just like 
with conventional otoacoustic emissions and middle ear 
reflex testing, the impact of such signals might produce 
movements of the eardrum that could be detected by 
microphones in the ear canal" (Lovich et al 2023 p2). 

EMREOs occur with eye movements, sometimes preceding 
them and continuing after for several tens of 
milliseconds. EMREOs provide precise information about 
the direction and amount of eye movement (Lovich et al 
2023). 

EMREOs have been found in rhesus monkeys and humans, 
which is not surprising as "the two species have similar 
visual and auditory acuity and similar eye movements. 
They are also known to integrate visual and auditory 
space in reasonably similar ways, showing similar 


° Discovered by Gruters et al (2018). 
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thresholds for fusing versus distinguishing visual and 
auditory locations" (Lovich et al 2023 p2). 

The exact role of EMREOs in the co-ordination of 
information "has not yet been established" (Lovich et al 
2023 p7). 


1.3. TOP DOWN FACTORS 


Newell et al (2023) began: "Object categorisation is 
a fundamental cognitive ability that allows us to 
efficiently recognise and interact with the content of 
our environment despite variability in the encoded object 
properties from one instance, or from one example, to the 
next. The processes underpinning category formation are 
thought to be mediated by both sensory-driven information 
and prior knowledge from memory, although how object 
categories emerge based on multi-sensory features is 
poorly understood" (pl). 

Concentrating on prior knowledge, "predictive coding 
models" propose that "best estimates" of objects (eg: 
templates) are stored in memory, which can be updated. An 
alternative is "statistical or Bayesian models", which 
argue that "the brain combines information from different 
senses in an optimal manner, based particularly on the 
inferred causality or reliability of the information from 
each modality" (Newell et al 2023 p3) (ie: a combination 
of both bottom-up and top-down processing). 

One problem is understanding how object categories 
form in the first place. This includes the study of 
infants and early development, and how categories ar 
organised in memory. 

In terms of research, Broadbent et al (2018), for 
example, presented 6-10 year-olds with familiar objects 
in auditory only, visual only, and audio-visual 
conditions. For all ages, recognition was best in the 
combined condition, but performance improved with age. 
Such research is taken to suggest that "improved sensory 
and perceptual precision with development likely 
underpins multi-sensory integration for categorisation" 
(Newell et al 2023 po). 

Studies with adults involve novel objects, and how 
they are categorised together (eg: Carvalho et al 2021). 


They show how information from different senses can "act 
as the 'glue'" ( Newell et al 2023 p8) in object 
categorisation, but also where information from one sense 


is uncertain. "Thus, hearing a bark will help recognise 
the shape of a dog if visual information is compromised 
(ie: if the image of the dog is partially occluded or 
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blurry)" (Newell et al 2023 p8). 


1.3.1. Meta-Cognition 


Maynes et al (2023) distinguished three types of 
multi-sensory experience: 


i) Congruent signals - "a single source in the 
environment giving rise to sensations in two or more 
modalities at the same time. For instance, when you talk 
to another person, you see their lips move and hear the 
sound of their voice, and this information arises from 
one source" (Maynes et al 2023 p2) (appendix 1B). 


ii) Integrated signals - the brain receives 
conflicting information but integrates the information as 
a single source (eg: ventriloquism illusion). 


iii) Segregated signals - information is received 
from more than one source, and the brain processes them 
as separate sources. 


Maynes et al (2023) compared the first two types 
using the "sound-induced flash illusion". This has two 
versions - the "fission" version ("if observers are 
presented with two brief beeps and one visual flash, they 
often report seeing two visual flashes"; p2), and the 
"fusion" version ("if observers are presented with one 
beep and two visual flashes, they sometimes report seeing 
one visual flash"; Maynes et al 2023 p2). 

Forty-six US undergraduates were presented with 
combinations of 0-2 flashes and 0-2 beeps in unisensory 
(eg: one beep only) and bisensory conditions (eg: 2 
beeps/1 flash) (table 1.2). The presentation of the 
information was rapid (eg: 50 ms between flashes), and 
the participants performed six blocks of 40 trials. 
Participants rated their confidence level of accuracy 
after each trial. 

Accuracy in perception was higher for unisensory and 
congruent bisensory conditions (eg: 2 beeps/2 flashes), 
and lower for bisensory illusion/incongruent conditions 
(eg: 2 beeps/1 flash) °®. Confidence level also showed the 


° Zaidel and Salomon (2023) asserted: "Put simply, there is no, nor has there ever been, true unisensory 
stimulation" (p2). They explained this statement: "Even when one’s eyes are fixated, and the head is 
stationary, proprioceptive signals from the eye muscles and vestibular signals regarding head position 
continue to bombard the brain. In complete darkness, the visual cortex remains highly active (and this 
activity even differs with eyes open or closed. Interoceptive and tactile signals are always present. 
Thus, ‘unisensory’ perception is a misnomer. In ‘unisensory’ experiments (or measurements), signals 
from the sense being probed are always received in combination with signals from the other senses and 
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same pattern. This suggested that confidence level, which 
was taken as a measure of metacognition, helps in 
distinguishing between congruent and illusory/incongruent 
multi-sensory information. 

Maynes et al (2023) proposed that metacognition 
(defined as "the capacity to monitor the quality and 
fidelity of one's own perception"; pl) relates to 
information-seeking. They stated: "Specifically, 
metacognition may link to information-seeking via some 
type of inverted U-function, wher xtremely high or 
extremely low confidence is associated with little 


a. 


information-seeking (if you know what something is, or 
information comes from an extremely noisy source, it may 
not be worthwhile to pursue further information), but 
intermediate levels of confidence may be linked to 
greater information-seeking to resolve ambiguities in 
stimuli. In this sense, perhaps lower levels of 
confidence for integrated multisensory stimuli could 
drive further information-seeking to determine whether 
the integrated signals truly came from a single source, 
or whether further exploration could lead to a more 


accurate inference about multiple sources of information 
being present" (Maynes et al 2023 pd). 


Condition Type Mean 
confidence 
(out of 4) 
1 beep (1B) Unisensory Sail 
2 beeps (2B) Unisensory 3.269 
1 flash (1F) Unisensory 3.38 
2 flashes (2F) Unisensory 3221 
1 beep/1 flash (1B1F) (Congruent) Bisensory 3.38 
2 beeps/1 flash (2B1F) Bisensory 3.06 
(Incongruent/"Fission" illusion - 2 
flashes perceived) 
1 beep/2 flashes (1B2F) Bisensory 34205 
(Incongruent/"Fusion" illusion - 1 flash 
perceived) 
2 beeps/2 flashes (2B2F) (Congruent) Bisensory 3/22 
Table 1.2 - Different experimental conditions of Maynes 


et al (2023). 


interpreted within that multisensory context. Thus, while we may decide to manipulate sensory signals 
and ask for decisions in one sensory domain in our experiments, we are always in practice impacting 
multiple sensory streams" (Zaidel and Salomon 2023 p2). 
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1.4. UNCERTAINTY 


Multi-sensory processing involves combining sensory 
cues from the same event, but distinguishing cues from 
different events. This is the "causal inference problem" 
(Marly et al 2023). To deal with the uncertainty, one 
hypothesis is that the brain has "conflict monitoring 
mechanisms" (eg: in anterior cingulate cortex) as part of 
the "cognitive control network" (Marly et al 2023). 

Certainty is greater when the sensory cues are 
clearly from the same event or show obvious disparity 
(ie: from different events). It is the "grey area" in 
between where uncertainty is higher, and this is studied 
in experiments. For example, the "ventriloquist illusion 
paradigm" ’, where "light and sound are presented from a 
range of locations, and participants have to locate 
either one or both stimuli. Sound location is often 
"captured' by the light (ie: the sound is perceived as 
coming from, or near, the position of the light) when 
spatial disparity is small" (Marly et al 2023 p2). 

Marly et al (2023) measured th lectrical activity 
of the brain (using electroencephalography; EEG) in 
Situations of perceptual conflict using the above 
paradigm. Visual and auditory stimuli were presented from 
five different positions, and the task was to state the 
location of the stimuli as quickly as possible. There 
were unisensory conditions (ie: sound or vision only) and 
a multi-sensory condition. Reaction time was quicker when 
the visual and auditory stimuli came from the same 
position or clearly from different directions. Reaction 
time was slower when there was uncertainty, and 
particular electrical activity (increased theta waves) 
was measured, "both indicative of conflict" (Marly et al 
A023 pl). 

The findings fit with previous studies that suggest 
a series of stages in multi-sensory processing. Marly et 
al (2023) explained that "multi-sensory perception would 
begin with the computation of unisensory estimates in 
sensory areas during the first 150 ms, followed by a 
fused estimate in posterior association brain regions, 
100 ms to 260 ms, and finally the computation of the 
causal inference estimate that would involve anterior 
regions starting as soon as 200-300 ms (timings vary 
between studies). The observed peak of theta in our study 
would fall between the timing of the generation of the 


7 Another audio-visual illusion used in research (eg: Meijer and Noppeney 2023) is the "McGurk- 
Macdonald illusion" (McGurk and Macdonald 1976). This is where incongruent sound and vision is 
perceived as an entirely different sound - eg: the sound "Ba" and lips seen making the "Ga" sound 
together is perceived as "Da". 
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fused estimate and the generation of the causal inference 
estimate. This timing fits well with the idea that the 
origin of the conflict signal is the arbitration between 
internal causal models, prior to th mergence of the 
final multi-sensory estimate" (pll). 


Multi-sensory information from vision and touch 
(haptic) about an object has been studied experimentally, 
for example by Badde et al (2023) Twenty-six student- 
participants were presented with objects of differing 
roughness or slant in three conditions - vision only, 
touch only, and combined vision-touch. The latter 
condition also included matched and unmatched versions 
(ie: seeing and feeling the same or different objects). 

There was great variety between individuals in the 
integration of visual-haptic information. "For example, a 
perceived mismatch between visual and haptic roughness 
would not deter the observer from integrating visual and 
haptic slant. These results indicate that participants 
based their perceptual judgements on a feature-specific 
selection of information, suggesting that multi-sensory 
causal inference proceeds not at the object level but at 
the level of single object features" (Badde et al 2023 
pl). 


Crossing a busy road is a good example to use to 
understand uncertainty in multi-sensory perceptual 
decision-making. The observer must perceive the oncoming 
traffic and estimate approach time in relation to the 
time to walk/run across the distance of the road. 
"Efficient decision-making requires accounting for 
sources of uncertainty (noise, or variability). Many 
studies have shown how the nervous system is able to 
account for perceptual uncertainty (noise, variability) 
that arises from limitations in its own abilities to 

ncode perceptual stimuli. However, many other sources of 
uncertainty exist, reflecting for example variability in 
the behaviour of other agents or physical processes" 
(Aston et al 2023 pl). 

So, uncertainty can be divided into internal/ 
intrinsic and external/extrinsic (Aston et al 2023). 
Internal uncertainty describes the limitations of the 
perceptual system (eg: vision and the amount of light as 
in crossing the road in near darkness), while external 
uncertainty refers to the environment (eg: the speed of 
individual drivers). "Human perceptual systems may be 
well equipped to account for intrinsic (perceptual) 
uncertainty because, in principle, they have access to 
this. Accounting for external uncertainty is more 
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challenging because this uncertainty must be learned" 
(Aston et al 2023 pl). 


Virtual reality allows participants to navigate 
around an environment which can be varied (eg: objects 
stationary or moving; self/observer stationary or 
moving). The mistakes in perception (ie: illusions) can 
be used to understand the process of perception. For 
example, accuracy in navigating around a virtual 
environment varies depending upon perceived self-motion, 
object-motion, and the apparent velocity of movement. In 
their experiments, Noel et al (2023) found that "humans 
are more likely to report moving targets as stationary at 
high velocities during concurrent self-motion. Similarly, 
during concurrent self- and object-motion, observers 
navigate to locations closer to the initial target 
location, as if the target were not moving" (p9). 

The virtual environment was manipulated such that 
the images on the retina could be due to self-motion, 
object-motion, or a combination of both. There is a high 
level of uncertainty and the brain is making decisions in 
continuous time. 


1.5. APPENDIX 1A - PREDICTIVE PROCESSING AND AESTHETICS 


Martindale (2007) observed: “How has it come to be 
that we know rather more how people remember nonsense 
syllables and far less than we should like to know about 
how they create and understand art?” (quoted in Leder and 
Pelowski 2023). 

From a "fleeting, messy, ever-changing bundle of 
light waves, sound waves and odour molecules" (pl) we 
make sense of the environment, but more than that, we 
make complex sense in what can be called aesthetic 

xperience (ie: beauty and art) (Frascaroli et al 2023). 

But trying to explain what we find beautiful has 
interested philosophers over history. The "classic 
theory" or "great theory" of beauty emphasises the 
"ordered arrangement of parts" - ie: "Ww xperience 
beauty whenever we perceive that different elements in 
our sensorium (the notes of a musical piece, the pigments 
on a canvas, the features of a landscape) stand in 
ordered relationships with one another and conspire, as 
it were, to create a meaningful whole — a structure or 
pattern that we can grasp. Experiences of beauty, in 
other words, would be moments of sudden clarity about the 
structure of our world: moments where everything 
makes sense, everything clicks into place, everything is 
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exactly as it should be, and more coherence, unity, 
consistency among disparate things is achieved. Artworks, 
in turn, would be objects specifically designed to afford 
these experiences" (Frascaroli et al 2023 p2). This idea 
has been taken up by contemporary neuroscience and 
psychology (eg: Ramachandran and Hirstein 1999) (or 
"neuro-aesthetics") - "the brain rewards progress toward 
organising the perceptual field into a meaningful 
configuration" (Armstrong and Detweiler-Bedell 2008 
quoted in Frascaroli et al 2023). 

An alternative idea on cognitive science is based on 
predictive processing (PP) (or predictive coding theory). 
Basically, the brain is testing hypotheses about the 
incoming signals to the senses to reach a conclusion 
about what is out there. "Prediction errors" help steer 
the brain to a better prediction. "The oscillations in 
uncertainty in its hypotheses about the structure of its 
world are therefore direct signals about how well it is 
doing in ensuring its continued existence as a viable 
model of that world. This means that perception and 
cognition are always soaked with affect, tied as they are 
with the hope that the world will reveal some structure 
and further our existence. What we call aesthetic 
pleasure, so the PP story suggests, is the positive 
affective feedback that we get when we are mor 
successful than usual in making sense of our environment 
(or, in PP terms, in reducing prediction error...). 
Aesthetic pleasure is, in other words, the mark of a 
cognitive and existential conquest" (Frascaroli et al 
20:23 p4ay. 


Frascaroli et al (2023) presented the case for a PP 
understanding of aesthetics, and introduced a special 
issue of the "Philosophical Transactions of the Royal 
Society B". 

One of the articles (Leder and Pelowski 2023) 
offered some caution: “We are happy that PP has opened 
exciting arguments in aesthetics, with suggestions that 
this might unlock how we perceive and respond to art. At 
the same time, however, while the promise of PP is slowly 
advancing from ‘biologically plausible’ to concrete, the 
movement from metaphor to actual mechanism appears even 
slower, especially in aesthetics research” (p2). 


A criticism of PP generally is the so-called “Dark 
Room Problem” (eg: Sun and Firestone 2020), which 
suggests that “if behaviour was driven only by the 
imperative to minimise uncertainty, organisms like us 
would always be found in situations that are minimally 
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uncertain and that preclude violations of our 
predictions” (Constant et al 2023 pl). This would also 
preclude creativity. 

Constant et al (2023) offered a solution that they 
called the “Enlightened Room Problem”. They explained: 
“Creativity emerges from various degrees of interplay 
between predictive brains and changing environments: ones 
that repeatedly move the goalposts for our own error- 
minimising machinery. By (co)constructing these 
challenging worlds, w ffectively alter and expand the 
space within which our own prediction engines operate, 
and that function as ‘exploration bubbles’ that enable 
information seeking, uncertainty minimising minds to 
penetrate deeper and deeper into artistic, scientific and 
engineering space” (Constant et al 2023 pl). 


1.5.1. Aesthetic Experience 


Van de Cruys et al (2023) began: “How to account for 
the powerful effects of art on us? For example, what do 
people mean when they insist that a piece of art (be it 
literature, visual art or music) has helped them through 
difficult times? Clearly, they do not mean this in the 
literal sense of ‘helping’: art does not help like anti- 
biotics do, eliminating the worldly cause of distress. 
Rather, art relieves distress by modifying our mental 
states — our construction of the brute facts of the world 
— and with that it opens up new opportunities for action. 
It provides the means for articulating, understanding 
and, ultimately, accepting or transforming our situation 
and ourselves” (pl). This is a way of describing the 
“aesthetic experience”. 

But how to pin down this experience? Shaviro (2012) 
described it this way in comparison to desire: “Desire is 
how the self projects itself into, and remakes, the 
world; aesthetic feeling is how the world projects itself 
into, and remakes, the self” (quoted in Van de Cruys et 
al 2023). 

Van de Cruys et al (2023) outlined four properties 
of the aesthetic experience based on the tension between 
receptivity (the willingness to change) and 
incomprehensibility towards the artwork: 


i) It is “a process, rather than a moment or 
instantaneous appraisal of a static thing” (Van de Cruys 
et al 2023 p2). 


ii) There are both positive and negative emotions. 
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iii) There are individual differences in what causes 
the experienc 


iv) It is an unrepeatabl xperience. “An aesthetic 
experience cannot be relived in the same way, because it 
consists in a change in one’s relation to the object that 
prompted it... This does not mean that the same work 
cannot give rise to new aesthetic experiences, but rather 
that..., these will be new instances caused by different 
generative processes sharing a similar structure” (Van de 
Cruys et al 2023 p2). 


Muth and Carbon (2023) used the term “Semantic 
Instability” (SeIns) to describe an experience “marked by 
a plurality of meanings, an unfulfilled potential for a 
determinate interpretation or even meaningful 
contradictions” (pl). SeIns “can generate aesthetic 
hedonics and interest” because art perception is 
different to problem-solving in that “we typically 
integrate contradictory elements dynamically and without 
the ultimate goal of resolving the contradictions” (Muth 
and Carbon 2023 pl). 


1.5.2. Aesthetic Appreciation 


The individual is faced with the anxiety of 
uncertainty (ie: a drive to avoid anxiety), but also the 
need to pursue new information for survival. Barbieri et 
al (2023) suggested that aesthetic appreciation creates a 
positive feeling that drives curiosity and so overcomes 
the anxiety of uncertainty. Barbieri et al (2023) put it 
thus: “We propose that aesthetic appreciation might have 
evolved as a self-generated intrinsic reward motivating 
us to tolerate arousing sensory uncertainty to favour the 
search for novelty, learning and change” (p3). 

These researchers reported two studies to support 
this view. In one of the studies participants listened to 
different types of classical music, and expressed an 
aesthetic judgment (AJ) about each piece. They also 
completed a measure of general anxiety afterwards. There 
was a negative relationship between AJ and anxiety score. 

In another study, fifty-one Italian adults watched 
magic tricks while listening to classical music or white 
noise. After each trick, the participants were asked to 
report their interest in the solution of the trick. Then 
whether they would play a gamble game to discover the 
solution, but receive a mild electric shock if lost. 
Willingness to gamble was classed as curiosity. The self- 
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reported level of interest in the solution did not vary 
between conditions, but participants in the classical 
music condition were more willing to gamble. 

Barbieri et al (2023) summed up: “These results were 
consistent with the idea that aesthetic appreciation 
could act as a ‘valve’, prompting the individual to 
perceive curiosity (ie: to consider novelty as a valuable 
opportunity to acquire new knowledge) rather than anxiety 
(ie: to consider novelty as a risk to be avoided)” (pl). 


1.5.3. Aesthetic Judgment 


Yoo et al (2023) concentrated on “aesthetic 
judgment”, which they saw as a “mixture of cognitive and 
affective processes... The cognitiv lements encompass 
recognition of basic elements of the artwork and beliefs 
about its meaning, the artist and the place of the 
artwork in art history and larger societal issues, as 
well as individual expectations and goals. The affectiv 
elements are feelings evoked by the art piece such as 
pleasure, beauty, wonder, boredom, doom or surprise” 
(ppl-2). 

These authors outlined four factors in aesthetic 
judgments (the first two are cognitive processes and the 
latter two are affective ones): 


i) The importance of the perceiver’s expectations. 


ii) Expectations about the fluency of a stimulus 
(ter sth ase of processing). “For example, take a 
stimulus like an androgynous human face. Note that it can 
serve as an excellent prototype of a broad category 
(human beings), but also as an atypical example of narrow 
gender categories (male versus female). Indeed, when 
participants’ task is simply to detect the presence of 
faces, the androgynous faces are fluent and liked. 
However, when participants’ task is to categorise faces 
into male or female, the very same androgynous face 
becomes disfluent and disliked” (Yoo et al 2023 p5). 


iii) The value placed upon the fluency of a stimulus 
- For example, if an individual wants to be surprised 
and/or uncertain, then fluency will be valued less. “For 
example, some people do not want to know the ending of a 
book from the beginning, they prefer a movie without a 
clear conclusion, they do not want to know the nature of 
love, and they appreciate an ambiguous artwork that 


remains puzzling over the years. An artist may prefer to 
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produce uncertain, disfluent works if their goal is to 


express indeterminacy, confusion, chaos or 
meaninglessness” (Yoo et al 2023 p7). 


iv) The stimulus in relation to the goals of the 
perceiver - “For instance, if the painting shows people 
enjoying fruits of immoral activities such as cheating at 
cards, viewers with just-world beliefs might not 
appreciate learning about it even if the craft is 
superb... Similarly, prude viewers may dislike learning 
more about nudes...” (Yoo et al 2023 p8). 


1.5.4. Specific Arts 


Concentrating on music, Cheung et al (2023) stated: 
“Apart from extra-musical factors such as episodic memory 
or contextual associations, an important intra-musical 
factor by which music itself induces pleasure in the 
listener is via the confirmation, violation and delay of 
listeners’ musical expectations” (pl). These researchers 
argued that “expectancy-driven musical pleasure results 


oad 


from two independent sources of expectancy: sensory 


as 


expectations arising from acoustic information in the 
auditory signal itself, as well as cognitive expectations 
derived from learned relations between musical elements 
abstracted from the auditory signal. While sensory 
expectations form over relatively short timescales, 
cognitive expectations are acquired after extended 
exposure to multiple examples of a musical style” (pl). 
One example of PP-based theory here is the 
“predictive coding of music” (PCM) model (eg: Friston 
2010). “In PCM, musical expectations generated from 
higher-order brain regions (eg: prefrontal cortex) are 
thought to propagate downwards towards lower-level 
sensory regions (eg: the auditory cortex). The 
discrepancy between expected and actual incoming signals 
in sensory regions results in a prediction error or 
surprise. This error signal is propagated upwards along 
the cortical hierarchy to refine future expectations. The 
gain of the expectation is modulated by a precision 
estimate, or the inverse of uncertainty. Precision- 
weighted prediction error signals are thought to 
constitute reward for the listener to continue learning 
towards generating more accurate future expectations” 
(Cheung et al 2023 p2). Support comes from studies (eg: 
Cheung et al 2019) that found that ratings of 
pleasantness of chords and melodies were “jointly 
predicted by their surprise and uncertainty in listeners” 
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(Cheung et al 2023 p2). 

Cheung et al (2023) performed similar research using 
thirty chord progressions sampled from commercially 
successful pop songs with two samples (25 adults (half 
musicians), and 39 adults). Ratings were taken of the 


surprise of each chord and the pleasantness. 


More research has been undertaken on PP and music 
than visual art (Kesner 2023). But applying PP to the 
latter, Kesner (2023) talked of a “general pictorial 
competence” that probably evolved in hominins, which 
depends on “regularities in the physical features of the 
world” (p6). This leads to the development of prior 
expectations, which are crucial in “a successful 
encounter” with art. “Psychological research has long 
established that people prefer medium levels of sensory 
uncertainty, and, in the context of art perception, a 
medium stimulus complexity” (Kesner 2023 p7). 


According to PP, the brain is “an engine of 
probabilistic hierarchical inference”, and Kukkonen 
(2023) argued that this can applied to “our engagement 
with literary texts”: “The notion of probabilistic 
hierarchical inferences seems to capture the way readers 
try to predict the verbal chain at different levels of 
abstraction, from sentences to characters’ choices and 
mental lives, to plot developments and what is likely to 
happen in a given genre. Literary texts, for their part, 
seem to provide a designed sensory flow that is carefully 
crafted to sustain and engage readers’ probabilistic 
inferences. They are ‘probability designs’: artefacts 
designed to ignite our inferential capacities with 
unexpected events (prediction errors), formulations that 
make the perspective from which the narrative is told 
salient (precision and attention), as well as signals 
genres (such as ‘crime fiction’ or ‘the modernist novel’) 
that best allow readers to make overall predictions 
(precision expectations)” (ppl-2). 


1.6. APPENDIX 1B - SOCIAL COMMUNICATION 


"Social communication depends on the integration of 
sensory, emotional and cognitive information by a large 
network of brain regions. We employ and integrate vocal 
sounds, mouth movements, facial motions and hand/body 
gestures, when speaking to one another. During 
communication, we must retain and integrate thes 
multiple auditory and visual cues in memory in order to 
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organise goal-directed behaviours. Thus, a communication 
circuit must include brain regions involved with 
perception, integration, memory and decision making 
regarding audiovisual information. The ventral frontal 
lobe, or inferior frontal gyrus, which includes Broca’s 
area, has been an essential player in audio-visual speech 
perception and verbal working memory" (Romanski and 
Sharma 2023 pl). 

Research on the "communication circuit" has involved 
non-human primates - for example, lesions of specific 
brain areas to establish function (eg: Goldman and 
Rosvold 1970), or recording of electrical activity in 
response to stimuli (eg: Diehl and Romanski 2014). 

In terms of behaviour studies, Hwang and Romanski 
(2015) trained rhesus monkeys to watch an audio-visual 
movie clip, and then press a button if the following clip 
was different in any way. This is an audio-visual non- 
match-to-sample (NMTS) task, and it is a test of working 
memory primarily. It was found that "working memory 
performance accuracy was higher in trials where subjects 
had to detect a change to the visual face component, 
compared to trials when they detected a change to the 
auditory component in the non-match period, indicating 
better memory for the visual stimulus" (Romanski and 
Sharma 2023 p5). 
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2. CHANGING MEMORIES 


.1. Creating false memories 
. Targeted memory reactivation 
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2.1. CREATING FALSE MEMORIES 


Johnson et al (2023) began: "The science of the 
fallibility of memory has demonstrated that our childhood 
memories do not necessarily provide a veridical 
representation of flawlessly recorded events as they 
occurred. Rather, our recollection of past events is 
susceptible to errors and biases and can be fragmentary, 
contain incorrect details, and, oftentimes, b ntirely 
fictional" (pl1011). 

The possibility of the creation of false memories is 
worrying. Wade et al (2002) showed this clearly ina 
classic experiment. Twenty participants (in New Zealand) 
were shown four pictures of their childhood events, of 
which one photograph had been doctored to show a 
fictitious hot air balloon ride. In three interview 
sessions the participants were asked to recall their 
memories of the pictures. "By the third interview 
session, half of the participants (n = 10) reported 
detailed, event-specific memories of the balloon ride, 
even though it was verified prior to the exposure that 
they had never experienced such an event" (Johnson et al 
2023 pl1011). 

The researchers involved in this study provided 
replications (eg: Garry and Wade 2005; Strange et al 
2008), but Johnson et al (2023) noted few independent 
replications, which motivated their research. 

Twenty young adults were recruited in Norway. The 
same gender distribution as Wade et al (2002) (50% male), 
same age range, and inclusion criteria (eg: not studied 
psychology). The fictitious childhood event was changed 
to a Viking ship ride (more culturally relevant to the 
participants). Family members provided information about 
the childhood events. 

Bach participant provided photographs of themselves 
at events while four to eight years old. One of the 
photographs was doctored to show a Viking ship ride. Each 
participant completed three interview sessions over 1-2 
weeks, during which they were encouraged to recall 
information about four photographs (three true and one 
altered). The interviews were transcribed, and three 
independent raters coded for false memories of the Viking 
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ship ride. 
Around 90% of true event information was recalled 
(based on information from family members). in terms of 
the false event, "one participant reported clear false 
memories, three participants reported partial false 
memories during the first interview (20%), six 
participants were trying to recall, and 10 participants 
had no memory of the fictitious event. In Wade et al 
(2002), 7 of the 20 participants (35%) remembered the 
alse event either partially or clearly during the f 


PT oa 


first 
interview, of whom six participants reported partial 
false memories and one participant reported clear false 
memories" (Johnson et al 2023 pl1014). In the third 
interview, eight participants (40%) recalled the false 


Q 


event partially or clearly compared to 50% in Wade et al 


- 


(2002). Table 2.1 gives two examples of false memories. 


e Fully-fledged false memories in first interview - "This boat 
gives me a weird feeling. I am a bit uncertain where this is. 
It must be a school trip in elementary school, second grade, I 
would guess when looking at the picture. The lifejacket looks 
familiar. [...] It looks cold... I don't remember anything 
else, just the feeling of being there, and the lifejacket that 
wasn't mine, but I borrowed it from someone" (pl1015). 


@e Uncertain in first interview, but more definite in third 
interview - "I thought it was really cool to learn about 
Vikings, and to try —- I remember that we had, let's call ita 
battle on a patch of gravel, and we had foam swords or 
something, and pretended to fight and dressed up and got to try 
a Viking helmet and stuff" (p1016). 


Table 2.1 - Two examples of false memories in Johnson et 
al: W202 3) 


The findings of Johnson et al (2023) provided a 
replication of Wade et al (2002). The former researchers 
stated: "The vivid imagery of detailed, event-specific 
information from a substantial percentage of the 
participants in the replication indicates that we 
successfully managed to modify the culturally appropriate 
and functionally equivalent false target event, similar 
to that of the original study. Also, the comparable 
findings of the 20-year-old original study and the 
current replication are astonishing, given that the 
replication was conducted at a time when people are much 
more familiar with and exposed to manipulated images and 
editing techniques" (Johnson et al 2023 pl1017). 
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2.2. TARGETED MEMORY REACTIVATION 


Hu et al (2020) began: “The idea of manipulating 
memories and thoughts during sleep is fascinating for 
neuroscientists, psychologists, and the general public. 
Although the idea may sound like science fiction, the 
past decade has witnessed an increasing number of studies 
wherein memory processing is directly manipulated during 
sleep. By covertly administering sensory cues while 
participants are asleep, associated memories from recent 
learning can be reactivated and modified” (p268). This 
process is known as “targeted memory reactivation” (TMR) 
(Paller and Oudiette 2022) °. 

“Memories continue to change, even after initial 
encoding and between episodes of deliberate rehearsal 
(Hu et al 2020 p269). 

The reactivation of stored information in memory 
during sleep aids consolidation of that information. Rats 
who have learned a new maze before sleep show the same 
pattern of neural activation during sleep as during that 
learning, for example (Wilson and McNaughton 1994). 

Preferential sleep consolidation could aid which 
memories are consolidated. Rasch et al (2007) is seen as 
the first experiment to show TMR in humans by using 
olfactory cues. During learning specific odours were 
presented, and during post-learning sleep, the same 
odours were released. Recall of learning improved for the 
material associated with the odours presented during 
sleep compared to non-exposed material. 

Rudoy et al (2009), and Anthony et al (2012) used 
auditory cues associated with learning in experiments 
with humans. Oudiette et al (2013) developed this work, 
and established the potential of TMR. 

Sixty participants over four experiments were 
instructed to remember the unique locations of 72 objects 
on a computer screen. When an object was presented, an 
associated sound was played (eg: “dog” and “woof”). Half 
the objects were allocated low reward values in terms of 
recall and half high reward values. After learning, 
participants were allowed to sleep for ninety minutes in 
a darkened room. 

In Experiment I, no object sounds were played during 
sleep, while in Experiment II 18 sounds associated with 
low-value objects were played in the middle of sleep. 
Experiments III and IV involved no sleep after learning, 
either watching a movie (with no object sounds played) in 
the former or solving difficult mental tasks (but the 
sounds were played during this time) in the latter. The 


_ 


° The term “TMR” is attributed to Oudiette and Paller (2013) originally. 
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outcome measure (or dependent variable) was a memory task 
where the 72 objects were presented individually in the 
centre of the computer screen and the participants had to 
move it to its specific location. 


Across all four experiments, more high-value than 


low-value objects were correctly recalled. This fits with 
the view that “important information tends to be 
replayed, and replay tends to forestall forgetting” 


(Oudiette et al 2013 p6673). More low-value objects were 


recalled in Experiment II (auditory cues during sleep) 
than Experiment I (sleep only). The lowest recall of low- 
value objects occurred in Experiments III and IV (awake) 


30 


oa 


(figure 2.1). 


@ High-value objects 
m Low-value objects 


Experiment | Experiment I| Experiment Ill © Experiment IV 


(Data from Oudiette et al 2013 table 1 p6674) 


Figure 2.1 - Mean correct recall of objects in four 
experiments. 


TMR research has grown in popularity in the 2010s, 
and experiments have investigated different sorts of 
learning (eg: word association; spatial memory; grammar 
learning). Another variable in the experiments is the use 
of within-participant (or repeated/related) design (ie: 
TMR and non-TMR conditions both experienced by each 
participant) or between-participant (or independent) 
design (ie: different participants in the TMR and non-TMR 
conditions). Studies further vary in when the TMR occurs 
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during sleep (eg: rapid eye movement (REM) or non-REM 
(NREM) (Hu et al 2020). 

Hu et al (2020) performed a meta-analysis of 91 TMR 
experiments (in seventy-three articles). The following 
key conclusions were drawn: 


i) Overall sleep TMR was mor ffective than no TMR 
in terms of post-sleep recall of learning prior to sleep. 


ii) TMR during stages 2 and 3 NREM sleep was 
effective, but not during REM sleep nor waking. 


any 


iii) Different forms o 
improved by TMR. 


memory and recall were 


Hu et al (2020) ended: “Despite some inconsistent 
results from single studies, meta-analytical results 
provide compelling evidence that applying sensory cues 
during NREM sleep can reactivate associated memories and 
promote memory consolidation. TMR effects are found 
across a range of learning domains, including but not 
limited to declarative memory and skill learning” (p239). 


Table 2.2 outlines some of the key methodological 
differences between the studies in the meta-analysis. 
Hu et al (2020) noted these general issues: 


a) That “some tasks placed in one learning category 
may engage processing that depends on multiple memory 
systems operative concurrently. For example, artificial 
grammar learning and other types of statistical learning 
may involve both implicit learning and declarative 
memory” (Hu et al 2020 p238). 


b) Whether the learning is new (eg: nonsense 
syllables) or familiar (eg: everyday words). “The 
ffectiveness of TMR is generally contingent on prior 
learning and associations made with specific cue stimuli’ 
(Hu et al 2020 p238). 


7 


c) Explaining TMR - eg: the neural mechanisms 
involved. “Cueing may simply bias spontaneous 
reactivation..., but there may be important differences. 
Because neural signals that completely 
and unequivocally indicate memory reactivation during 
sleep have not yet been established, this question 
remains open” (Hu et al 2020 p239). 


d) Whether TMR improves the memory for cued 
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information, but harms other recent learning. “That is, 
memory storage may normally involve competition, such 


that enhanced storage of some information would be 
expected to have repercussions” (Hu et al 2020 p239). 


e) Almost all studies were laboratory-based 


experiments. Goldi and Rasch (2019) was an exception that 


By 


investigated TMR for vocabulary learning while sleeping 
at home. 


e Length of learning, sleep (eg: less than 1 hour to 8 hours), 
and retention periods. 


e Activity during waking condition. 


e Time of experiment during the day/night cycle. 


e Type of TMR - olfactory; auditory-non-verbal (ie: sounds); 
auditory-verbal (ie: spoken words). 


e Between-participant or within-participant design. 


e Type of memory studied. 


e Sample size and make-up (eg: age range of 13 to 71 years old). 


e Stages of sleep when TMR administered. 


e Outcome measure - eg: correct recall; errors; false memories; 
recognition; performance of skill. 


Table 2.2 - Key methodological differences between 
studies of TMR. 
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3.1. INTRODUCTION 


Humans are able to think about possibilities. "We 
can all form theories about how an effect might have been 
produced, even if we were not there to witness the actual 
cause ourselves. And we can all imagine and compare a 
multitude of possible ways the future might turn out, or 
speculate about once possible ways the past might have 
turned out but did not" (Redshaw and Ganea 2022 ppl-2). 

While Comrie et al (2022) began: "The ability to 
imagine is essential to human experience. At a broad 
level, imagination has a major role in human creativity, 
agency and everyday thoughts and actions. More 
specifically, humans have and express many types of 
imagined experiences. These include recollections, 
predictions, simulations, counter-factuals, fantasies, 
suppositions and mind-wandering — and, in pathological 
cases, hallucinations and confabulations. These wide- 
ranging forms of imagination are relevant, if not 
essential, to a similarly wide range of cognitive 
domains, such as memory, planning, learning and 


inference" (pl). However, these authors admitted that 
"our understanding of how imagination is realised as a 
biological process in the brain remains nascent" (Comrie 


et al 2022 pl). 


Redshaw and Ganea (2022) outlined the different ways 
of studying thinking about possibilities (TAP) and the 
different issues: 


e Neurophysiological basis. 
e Development of TAP in childhood. 
e Cultural differences. 
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e Advantages of TAP in terms of learning, and 
decision-making. 
e Non-human animals. 


e The evolution of TAP in humans. 


3.2. NEUROPHYSIOLOGICAL BASIS 


The "fundamental ability to generate possibilities 
that do not correspond to the actual present" (ppl-2), 
Comrie et al (2022) called "generativity". Researchers 
have tried to understand the neurobiology of such a 
process by distinguishing "actual from imagined 


— 


experience, specific neural processes in the healthy 
brain must ‘parse’ internal representations as ongoing 
experience (actual) versus internally generated 
alternativ xperience (imagined)" (Comrie et al 2022 
p2). 


Comrie et al (2022) reviewed the evidence based 
around five questions: 


a) Where in the brain does generativity occur? The 
answer seems to be the hippocampus. One line of evidenc 
is brain-injured human patients, like "HM". He had part 
of the hippocampus area surgically removed to deal with 
sever pilepsy, and was subsequently found to have 
problems in future-oriented thinking and constructing 
fictional events (eg: Klein 2002). Neuroimaging studies 
have provided more evidence, showing activation of the 
hippocampus during imagining tasks (eg: Addis and 
Schacter 2011). 


b) How to identify generative neural activity? 
Rodent studies are particularly relevant here. Electrodes 
can be attached to the brain which record neural activity 
during movement through a maze. "Place cells" in the 
hippocampus fire at particular places in the maze to 
suggest actual perception (appendix 3A). But there is 
evidence of place cells firing when a rat is not at that 
particular place, and "these moments can be provisionally 
understood as times in which a representation of the 
place field location is internally generated, even though 
the animal actually occupies a different location at that 
moment" (Comrie et al 2022 p3) (eg: Pavlides and Wilson 
1989). 


c) What brain patterns correlate with generativity? 
Using rodents also, studies have found a pattern of 
hippocampal cell activity during sleep after a day spent 
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running a new maze. "Firing sequences of place cells that 
were active during running on a maze were found to 
reactivate in similar sequential order during subsequent 
sleep, as if briefly 'replaying' past spatial experience" 
(Comrie et al 2022 p4) (eg: Nadsady et al 1999). 

But the pattern of firing can be more than a simple 
replay, suggesting "an abstract internal spatial model of 
the encountered environment, or a spatial ‘cognitive 
map'. For instance, replays can be biased toward paths 
that are less behaviourally traversed, and replays can be 
consistent with random trajectories through a familiar 
space; replays like these may sample locations that are 
not the most behaviourally salient or the most physically 
occupied to support the maintenance of a flexible model 
of the environment, and this function could help explain 
why replays are inconsistent with a rigid recapitulation 
that passively records recent experience. These reports 
suggest that replay, instead of directly reinstating 
specific episodes, may abstractly reflect past experience 
via an internal spatial map" (Comrie et al 2022 pp4-5). 

Place cell firing patterns have been recorded before 
starting a maze which suggests an anticipation of future 
events (Comrie et al 2022). 


d) How the brain organises actual versus generative 
patterns? There is evidence of organisation of cells in 
the rodent hippocampus that distinguishes "actual 
present" from imagined, but the latter sub-divides into 
"experienced past", "anticipated future", and 
"possibilities" (Comrie et al 2022). 


e) What is the role of generativity in cognition? 
One theory is that "the hippocampus is a system for 
‘relational' memory: a system for inferring abstract 
relationships between observable events (such as sensory 
stimuli, actions and internal states). Integrating 
information in relational memory is beneficial in that it 
enables inference and generalisation to novel 
circumstances, such as those wher lements of previous 
experience are reconfigured. This can be advantageous 
regardless of whether those novel circumstances can be 
anticipated at the time of generating the relational 
information, consistent with the idea that generative 
activity can but does not always relate to immediate 
behaviour" (Comrie et al 2022 pl0). 


Neuroimaging has also shown that particularly areas 
of the brain's "default mode network" (DMN) ° are active 


” This includes the medial prefrontal cortex, anterior cingulate cortex, lateral and medial temporal 
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during counter-factual thinking (ie: imagining 
alternatives). Khoudary et al (2022), for example, asked 
32 participants to imagine that negative past personal 
experiences turned out better, "either by envisioning 
that they had taken an alternative course of action or by 
envisioning that the contextual circumstances surrounding 
the event had been otherwise" (Redshaw and Ganea 2022 
p2). 


Redshaw and Ganea (2022) explained: "Distinct 
regions beyond the DMN, however, were differentially 
activated depending on the type of counter-factual 
imagined. Overall, this finding suggests that the human 
brain does not support an encapsulated, one-size-fits-all 
approach to imagining possibilities, and instead this 
capacity draws on a range of interlinked and 
differentially engaged neurocognitive components" (p2). 

This was a study of "episodic counter-factual 
thinking" (eCFT) (De Brigard and Parikh 2019), which can 
be envisaged this way: "We may imagine having answered an 
important phone call we actually missed, or maybe having 
said something we actually never did" (Khoudary et al 
2022 pl). 

Similar neural activity in the DMN was reported by 
Addis et al (2009) during episodic memory retrieval, 
episodic future thoughts, and eCFT. However, there are 
also different brain areas used in each case, and there 
are differences based on the type of eCFT, as Khoudary et 
al (2022) showed. This study distinguished between 
internal and external eCFT. The former is encapsulated in 
the phrase, "if only I had done something different", and 
the latter is "if only the environment had been 
different". For example, getting sunburnt on the beach: 
"Thinking back, you may consider having done something 
different: 'If only I had applied sunscreen before going 
to the beach!'. Alternatively, you could imagine that 
something about the situation itself had been different 
to prevent the negative outcome from happening: 'If only 
it had been cloudier that morning!'" (Khoudary et al 2022 


p2). 


Specifically Khoudary et al (2022) found: "Both 
internal and external eCFT engaged midline regions of 
cingulate cortex, a central node of the DMN. Most 
activity differentiating eCFT, however, occurred outside 
the DMN. External eCFT engaged cuneus, angular gyrus and 
precuneus, whereas internal eCFT engaged posterior 
cingulate and precentral gyrus" (pl). 


lobes (including hippocampus), inferior parietal lobule, and caudate (Khoudary et al 2022). 
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3.3. DEVELOPMENT OF TAP IN CHILDHOOD 


The early origins of TAP have been investigated with 
four years old being a cut-off point in such studies (ie: 
unable to show TAP younger than that; eg: Redshaw and 
Suddendorf 2016). While, Cesana-Arlotti et al (2022) 


(table 3.1), for example, suggested that fourteen month- 
olds could "Simultaneously represent two mutually 
exclusive possibilities" (Redshaw and Ganea 2022 p3), 


though behavioural and physiological measures have to be 
used. 

Another method used with 12 month-olds was the 
presentation of four balls on a screen (three yellow and 
one blue, say), which disappeared one at a time (eg: 
Teglas et al 2007). This was used to test probabilistic 
intuitions. It was found that "12-month-olds 
look longer at a less probable result (the blue ball 
exiting) than at a more probable one (a yellow ball 
exiting). Infants seem to expect the more probable 
outcome..." (Cesana-Arlotti et al 2022 p7). 

Redshaw and Suddendorf (2016) used an upside-down Y- 
shaped tube and dropped a ball in at the top. It could 
fall out in one of two possible directions, and the 
children had to catch it. "Strikingly, only four-year- 
olds prepared the optimal action reliably and 
spontaneously (simultaneously reaching for both exits 
with their hands). Instead, younger children and apes 
failed to do so, suggesting that they could not optimally 
prepare for two mutually exclusive outcomes. 
Interestingly, turning the task into a social game did 
not improve performance, and the challenge children may 
face seems not to be simply one of manual co-ordination" 
(Cesana-Arlotti et al 2022 p2). 

The contradictory findings may be a product of the 
methodological difficulties of testing young children, 
and that the different studies were testing slightly 
different aspects of TAP. For example, commenting on 
Redshaw and Suddendorf (2016), 

Cesana-Arlotti et al (2022) argued that failure in the 
task of catching the ball could be due to an inability to 
represent two mutually exclusive possibilities at once or 
"may reflect their limitations in decision-making when 
facing uncertain events" (p2) 7°. 


'° This task has been used with captive chimpanzees who failed to cover both exits. However, critics 
argued that chimpanzees did not cover both exits because the behaviour was not natural to them (Ly 
2023). Engelmann et al (2023) adapted the experiment to behaviour that wild chimpanzees were 
familiar with - namely, a platform containing food that would become unstable and the food would be 
lost. In the control condition when a rock was dropped on the platform it was obvious which side to 
hold to stop the food being lost. In the experimental condition (with the equivalent of the Y-shaped 
tube), it was not clear which side of the platform to hold, but thirteen of fifteen chimpanzees tested 
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e Cesana-Arlotti et al (2022) used infants' pupil dilation as the 
measure of the ability to represent multiple possibilities. Ten 
and 14 month-olds were presented with three objects (a puppet, 
a toy elephant, and a ball) before they were hidden behind 
screens (the puppet and the ball behind one screen and the toy 
elephant behind its own screen). Then one of the three objects 
was partially revealed in such a way that it showed the common 
feature to all three objects. In the l-possibility condition 
(control) the object emerged from the screen hiding one object, 
while in the 2-possibility condition, the object came from 
behind the two-object screen. 


e cCesana-Arlotti et al (2022) explained the logic behind the use 
of the pupil dilation measure: "Just as adults' pupil diameter 
grows monotonically with the amount of information held in 
memory, we expected that infants' pupil size would increase 
with the number of alternatives sustained in memory as 
candidate identities for the partially occluded object" (pl). 


e For 14 month-olds only, "their pupils increased more at the 
appearance of the partially occluded object when there were two 
possible identities" (Cesana-Arlotti et al 2022 p7). This was 
taken as evidence that the infants could hold two possibilities 
in their mind at once. 


Table 3.1 - Cesana-Arlotti et al (2022). 


Using pretend play and imagination as evidence, 
Harris (2022) argued that 2-3 year-olds "can imagine and 
draw appropriate inferences about possible events and 
Situations" (Redshaw and Ganea 2022 p3). 

During pretend play young children show evidence of 
enacting spatial displacements (eg: pouring tea from an 
empty teapot), for instance. This has been shown in 
experiments with two year-olds by Harris and Kavanagh 
(1993). "For example, having watched a play partner 
‘pour’ make-believe tea over one of two pigs and been 
prompted to 'dry the pig who's all wet', they used a 
towel to wipe the appropriate pig, even though neither 
pig was actually wet" (Harris 2022 p2). 

This was Experiment 5 by Harris and Kavanagh (1993), 
while in Experiment 6 2-3 year-olds were asked about the 
imagined events. "Having watched Teddy, a mischievous 
hand puppet, enact various transgressions (eg: 'pour' 
pretend tea from an empty teapot over the head of a toy 
monkey), they replied appropriately to an adult's 
questions about what had happened. When asked what had 
been displaced (What did Teddy put on the monkey's head?) 
and about the resultant state of Teddy's victim (Is the 
monkey's head wet or dry?), the majority of both younger 


were more likely to hold both sides (Ly 2023). 
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(24-30 months) and older (25-36 months) children replied 
correctly (ie: 'tea' and 'wet')" (Harris 2022 p2). 


Wente et al (2022) investigated pretend play and 
counter-factual thinking in an experiment with 62 three 
and four year-old Peruvian children, and fifty-seven US 
children from low-income households. The experimental 
design was based on Buchsbaum et al (2012), and involved 
a "zando" machine. This is a wooden box with a doorbell 
inside that plays "Happy Birthday" if the correct block 
("zando" vs "non-zando") is placed in the box. Child 
learned the causal relationship that the zando block led 
to the tune playing. 

Then the children were asked counter-factual 


questions (eg: "If this one (while pointing to the zando) 
was not a zando, what would happen when we put it on the 
machine?"; p5). This was the control condition of the 


experiment. In the subsequent condition, the machine was 
taken away and the children were given wooden blocks. 
They were encouraged to pretend that the blocks were the 
zando machine, and asked the same counter-factual 
questions (pretend play condition). 

Both groups of children were able to answer the 
counter-factual questions correctly significantly above 
chance in the pretend play condition, but not in the 
control condition. 

Buchsbaum et al (2012), using sixty US children from 
mixed-income households of the same age, had found that 
both conditions were answered correctly above chance. 
Wente et al's (2022) study showed both socio-economic 
status/income, and cultural differences in understanding 
counter-factual reasoning in young children. 


Gautam et al (2022) concentrated on regret. They 
stated: "Regret is an emotion that arises when we compare 
what is happening in the present to what would be 
happening had an alternative course of events taken place 
in the past. For example, when caught in a downpour you 
might feel regret if you realise that you did not pack an 
umbrella in the morning. Regret thus presupposes a 
capacity for counter-factual thinking: reflecting on a 
past event and understanding that there were once other 
ways that event might have turned out. Regret has 
important functions, such as helping us learn to behave 
more adaptively when confronted with similar situations 
with multiple possible outcomes in the future. In fact, 
evidence suggests that adults explicitly identify regret 
as the most beneficial negative emotion, in terms of both 
learning from past mistakes and making better future 


pa 
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choices" (Gautam et al 2022 pl). 

What about children? The "two boxes task" is 
commonly used to study it. A child is presented with a 
choice of two boxes containing prizes, and must choose 
one. It contains a small prize (eg: a sticker) and child 
is asked how they feel. Then they are shown the prize in 
the unchosen box (which is always better) (eg: a toy), 
and asked how they feel now. Six year-olds show evidence 
of negative emotions after seeing the non-chosen prize 
(eg: Amsel and Smalley 2000). 

Some researchers interpret this emotion as regret, 
but Gautam et al (2022) were not convinced: "After all, 
the children, upon seeing the alternative prize, may 
Simply feel another negative emotion, such as frustration 
(eg: 'Oh no! I want that other prize!'), without 
considering and mentally undoing their past choice (ie: 
"Oh no! I wish I had chosen differently!'). Therefore, it 
is unclear if children in such tasks are genuinely 
experiencing the counter-factual emotion of regret" (p2). 

Gautam et al (2022) added different conditions to 
their version of the two boxes task. One hundred and 
Sixty 4-9 year-olds were allocated to one of four 


conditions when faced with two wooden boxes containing 
prizes (one or five stickers): 


1. "Better-outcome-possible" condition - Children 
chose between the boxes, but always received the lesser 
prize. This should produce counter-factual regret when 
shown the better prize in the non-chosen box. 


2. "Better-outcome-impossible" condition - The 
children were forced to choose one box which contained 
the one sticker. This should not produce counter-factual 
regret as there was no genuine choice. 


3. "Worse-outcome-possible" condition - The same as 
(1) but the children always got the better prize. This 
should produce a feeling of relief after seeing the prize 
in the other box. 


4. "Worse-outcome-impossible" condition - A mirror 
of (2) but with the better prize. 


After the choice, the mean emotion rating was six on 
a seven-point scale in all conditions. The rating offered 
after seeing the alternative prize was "happier", 
"sadder", or "the same". Children were sadder in the 
better-outcome conditions, and happier in the worse- 
outcome conditions. "Results showed that children in the 
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better-outcome-possible condition were significantly more 
likely than those in the better-outcome-impossible 
condition to report feeling sadder when the better prize 
was revealed..., consistent with the notion that children 
were genuinely experiencing the counter-factual emotion 
of regret in the former condition" (Gautam et al 2022 
p5). There was no difference found based on age, which 
was unexpected. 

There was no evidence of counter-factual relief in 
the worst-outcome conditions. 


More sophisticated abilities related to TAP develop 
later in childhood, and may be linked to language 
development. The evidence is divided about possibility 


reasoning (or modal cognition) (eg: "if I do X, Y will 
happen"). In the debate, "findings from studies with 
infants and non-human animals are interpreted as evidence 


= 


for successful logical reasoning in the absence of 
language, while findings from studies with pre-schoolers 


point to persistent difficulties" (Grigoroglou and Ganea 
2022" PPL=2)-s 

For example, children are shown two containers, of 
which one holds an unseen reward. Th xperimenter shows 


that the reward is not in container A, say, and then asks 
the child to find the reward. Pre-linguistic children and 
non-linguistic animals choose container B (eg: Call and 
Carpenter 2001). This would suggest an understanding of 
the logical reasoning "not A, therefore B". However, this 
is not necessarily the case. "For instance, when shown 
that one of two locations is empty, children may engage 
in an ‘avoid empty' strategy and search in the 
alternative location not because they have reached th 
logical conclusion that the reward necessarily has to be 
in the alternative location but because it is the only 
salient hiding location (after avoiding the empty 
location)" (Grigoroglou and Ganea 2022 p3). 

Mody and Carey (2016) created a more sophisticated 
version of the task with four containers, but the reward 
could by in A or B, say. Children were shown that 
container A was empty, for instance. The use of logical 
reasoning would lead to choosing B, but the "avoid empty" 
strategy would mean choosing B, C or D. Three year-olds 
and above chose B significantly above chance, but younger 
children did not (Grigoroglou and Ganea 2022). 

Grigoroglou and Ganea (2022) linked the development 
of possibility reasoning with the acquisition and use of 
possibility terms (eg: maybe, possibly, probably, might, 
could be). Basic abilities appear at three years old, but 
a "mature understanding" later (eg: seven years old) 


Psychology Miscellany No. 202; June 2024; ISSN: 1754-2200; Kevin Brewer 


41 


(Grigoroglou and Ganea 2022). 

Grigoroglou and Ganea (2022) concluded that 
"children's conceptual understanding of logical and modal 
concepts seems to go hand in hand with children's 
linguistic understanding of logical and modal terms. 
However, evidence from children's production shows that 
such terms tend to appear in speech earlier than the 
age children comprehend these terms and before they show 
evidence for mature logical and modal reasoning" (p7). 


"People often speculate about what the future holds. 
They wonder what will happen tomorrow, and what the world 
will be like in the distant future. Nonetheless, people's 
ability to consider future possibilities may be 
restricted when they consider their own futures. Adults 
show the 'end of history’ illusion, believing they have 
changed more in the past than they will in the future" 
(Goulding et al 2022 pl). This is a deficit in TAP here 
linked to being anchored in the present (Goulding et al 
ZO ZA); 

Goulding et al (2022) investigated this behaviour 
with three year-olds in Canada. The children were offered 
pairs of objects - one appropriate to a child or baby, 
the other an adult (eg: toy vs wrist-watch). They were 
asked which they preferred now, in the past as a baby, 
and in the future as an adult. The children were better 
at imagining their preference in the future if they had 
not been reminded of the present preference - ie: "when 
they are not confronted with things that anchor them to 
the present" (Goulding et al 2022 p7). 

The researchers drew the conclusion that "children's 
difficulties with mental time travel reflect a failure to 
shift away from the present rather than an inability to 
Simulate alternative possibilities" (Goulding et al 2022 
pl). However, the researchers did accept that the 
explanations are not necessarily mutually exclusive. 


3.4. CULTURAL DIFFERENCES 


Human cultural practices can enhance TAP; for 
example, "cultural time-keeping methods such as oral or 
physical calendars can allow us to mentally generate and 
communicate about possibilities that might take place at 
precise and agreed upon moments in the future" (Redshaw 
and Ganea 2022 p4). 

Vale et al (2023) preferred the term "prospective 
cognition", and reflected on its link to culture. Culture 
involves many behaviours that have future benefits, 
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including food preservation, avoiding certain toxic 
foods, and "healthy habits". 

Prospective cognition (or prospection), according to 
Szpunar et al (2014), involves "several modes of future- 
oriented thought including planning (pre-empted steps to 
meet a goal), prediction (estimated likelihoods), 
Simulation (mental representations of the future), and 
intention (mental act of goal setting). Each of these 
modes can include content that is episodic (relating to a 
specific personal experience), semantic (relating to 
general knowledge of the environment) or a mixture of 
both" (Vale et al 2023 pp2-3). 

A two-way relationship between prospective cognition 
and culture can be described. Prospective cognition 
allows culture to develop the behaviours for the future 
benefits, but culture gives the opportunity for 
prospective cognition. "At the most basic, of course, 
culture allows the time and space to develop future 
goals. Routine tasks, such as planning routes to work, 
involve many cumulative inventions across generations 
that yielded pathways, trains, buses and cars. In many 
cases, the advent of these cultural products (eg: 
agriculture and institutions that trade in consumables) 
alleviate present motivations, such as the need to forage 
or to cultivate produce. With basic needs met, a shift 
towards prospective cognition and the pursuit of goals in 
the distant future can occur, spurring modes of future 
thought" (Vale et al 2023 p7). 


en: 


Cultural influences may also bias and limit TAP. For 
example, Epstude et al (2022) asked highly partisan 
Democrat and Republican voters in the USA to think of 
alternative scenarios during the time of the Trump or 
Biden presidency. Participants were better at the 


counter-factual thinking if it aligned with their 
political views (appendix 3B). 


3.5. ADVANTAGES OF TAP 


In terms of decision-making, TAP is involved in the 
Situation like the offer of "£5 now or £10 in one week". 
Previous research has suggested that delaying the reward 
was a self-control issue, but Bulley et al (2022) (table 
3.2), for instance, found the value placed on the reward- 
time combination was more important (eg: subjective value 
of anticipating the pleasure of a future reward). 
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e Study - Choosing to delay gratification for larger, later (LL) 
rewards compared to smaller, sooner (SS) rewards (inter- 
temporal choice task). 


e Participants - 117 adults (mostly in USA) recruited online. 


e Task - "Monetary Choice Questionnaire": varying choices of SS 
vs LL US dollar rewards (eg: $18 today or $25 in nineteen 
days). After each choice, there was a rating of the confidence 
of the decision as the right one. 


e comment - If the choice of a LL reward was a sign of self- 
control, then participants should be more confident about this 
decision compared to a SS reward choice. This was not found. 
The confidence level varied with the length of time, and the 
size of the SS compared to the LL rewards. The researchers took 
such findings as evidence of decision-making based on 
subjective valuation of the options. Bulley et al (2022) did 
not reject self-control completely in delaying of gratification 
choices, simply that it is one factor in such decisions. As 
they stated: "simply interpreting choices of LL reward as a 
capacity for self-control is problematic" (p8)... "Our findings 
challenge self-control views and instead cast inter-temporal 
choice as a form of value-based decision-making about future 
possibilities" (Bulley et al 2022 pl). 


Table 3.2 - Bulley et al (2022). 


Causal judgments involve the attribution of event A 
causing event B, say. There are two possible ways that 
causal judgments are made, according to Gerstenberg 
(2022) (who favoured the first in the the "counter- 
factual simulation model"; Gerstenberg et al 2021) - 


£ 


counter-factual simulation or hypothetical simulation. "A 
counter-factual simulation involves observing what 
actually happened, mentally travelling back in time to 
imagine a change to what actually happened, and then 
Simulating how this alternative possibility would have 
played out. If the outcome in the counter-factual 
Situation would have been different from what actually 
happened then the event of interest caused the outcome. 
By contrast, a hypothetical simulation involves imagining 
a possible future. This does not require going back in 
time and mentally changing something that already 
happened" (Gerstenberg 2022 pl). 


TAP includes an element of "moral judgments of 
"permissibility', like whether an agent 'ought' or 
"should' do something" (Acierno et al 2022 pl). These 
moral judgments form "implicit or default (rather than 
deliberative) representations of what is possible" 
(Acierno et al 2022 pl). 
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Reasoning is often called "modal" thought, and it is 
connected to moral thought. "For example, people's 
judgements of morality influence their perceptions of the 
relevance of alternative possibilities, such that they 
think it is more relevant to consider morally good 
possibilities than morally bad ones. Moreover, prior work 
by Shtulman & Tong [2013], found that the more frequently 
participants judge extraordinary events as possible, the 
more often they judge extraordinary actions as 
permissible" (Acierno et al 2022 p2). 

How quickly a decision must be made is also 
important, and Acierno et al (2022) factored this 
variable into their study. US adults were recruited via 
"Amazon Mechanical Turk" or "Prolific", and they were 
presented with the scenario of a cake containing nuts 
baked for a children's party, but at the very last moment 
it is discovered that one of the children has a nut 
allergy. Options of what to do were offered, which were 
"ordinary" (eg: serve another food instead), "impossible" 
(eg: "zap the nuts from the cake"), "immoral" (eg: "lie 
that the cake is nut-free"), "improbable" (eg: find a 
nut-free cake at hand), and "irrational" (eg: postpone 
the party for one month). Time to respond was varied 
(slow or fast). In total, varying the options, there were 
over 100 000 moral judgments by around one thousand 
participants. 

Acierno et al (2022) summed up the findings: "When 
making moral permissibility judgements quickly, 
participants were more likely to judge that improbable, 
irrational and impossible actions were not permissible, 
indicating that default representations of permissibility 
may be reflecting default representations of possibility" 
(pl). The study supported a connection between modal and 
moral thought. 


Fitzgibbon and Murayama (2022) described the concept 
of "counter-factual curiosity" - the motivation to seek 
out "information about what might have been had past 
events been different" (p2). Such a behaviour can lead to 
negativ xperiences (eg: regret), but Fitzgibbon and 
Murayama (2022) argued that "it is motivated by the 
potential use of counter-factual information for adaptive 
decision making (its long-term instrumental value) and 
the drive to reduce uncertainty" (p2) (eg: "Functional 
Theory of Counter-factual Thinking"; Epstude and Roese 
2008). 

There are two types of the behaviour - "spontaneous 
counter-factual thoughts" (ie: thinking about 
alternatives), and seeking out counter-factual 
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information. The former is seen in an experiment where 
participants were asked to think aloud while playing a 
card game (Markman et al 1993). Nearly all participants 
had spontaneous thoughts like, "If I had gotten the king, 
I would've lost to the dealer" (quoted in Fitzgibbon and 
Murayama 2022). 

Another example is Davis et al's (1995) study of 
individuals who had experienced traumatic life events 
(eg: sudden death of an infant). "The majority of 
respondents (76%) reported mentally undoing the event in 
the weeks after an infant's death, and almost half 
reported still thinking 'if only...' thoughts years after 
the event" (Fitzgibbon and Murayama 2022 p2). 

In terms of seeking out information about counter- 
factual alternatives, research has found the behaviour by 
adults, children, and non-human primates. The use of 
vignettes is a common method of study with humans. For 
example, Shani and Zeelenberg (2007) "asked participants 
whether they would want to know the outcome of a lottery 
for which they had lost their ticket (but remembered th 
numbers). Participants were more likely to want to know 
when there was a high likelihood that they had had the 
winning numbers than when there was a low likelihood, 
suggesting that their curiosity was sufficient to 
overcome the potential regret of not returning the 
ticket" (Fitzgibbon and Murayama 2022 p2). 

An example with children is Fitzgibbon et al's 
(2019) card game with four and five year-olds who had the 
opportunity to use "magic glasses" to peak at an unseen 
card after a choice. Three-quarters of the children chose 
to know, and more so after a negative decision 
(Fitzgibbon and Murayama 2022). 

Rhesus macaques have been studied among non-human 
primates. For example, in a study by Wang and Hayden 
(2019), "macaques made choices between probabilistic 
gambles for water rewards. Importantly, choosing one of 
the gambles would additionally lead to information about 
the outcome of the unchosen gamble. The macaques chose 
the informative gamble more often than would be expected 
by chance, even foregoing water rewards to do so: they 
chose the informative gambl ven when the chance of 
reward (expected value) was lower, suggesting that the 
information itself had some value" (Fitzgibbon and 
Murayama 2022 p3). 


3.6. NON-HUMAN ANIMALS 


Delay of gratification shows the ability to imagine 
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the future, and it is also a sign of self-control. "Self- 
control is cognitively challenging as individuals must 
not only resist temptation in the present moment but also 
override the tendency to temporally discount or devalue 
future rewards" (Schnell et al 2022 pl). Is there an 
association between self-control and other cognitive 
abilities related to intelligence? 

In humans, children who resist temptation on the 
"marshmallow test" (ie: one marshmallow now or two in a 
few minutes) (Mischel et al 1989) subsequently do better 
in academic tests (Schnell et al 2022). 

Among chimpanzees, greater self-control is 
associated with better performance on thirteen different 
cognitive tasks (Beran and Hopkins 2018). This 
relationship has been observed in other species, like 
corvids (crow family). 

There is a logic that these birds should show such 
behaviours, as Schnell et al (2022) explained: "In 
corvids, caching behaviour, ie: hiding food for later 
consumption, offers an illustrative example of how 
ecological pressures such as the need to inhibit 
immediate gratification to plan for future meals may have 
driven the evolution of self-control. Caching behaviour 
also imposes social pressures that might have reinforced 
the evolution of self-control. For instance, some species 
of corvids such as crows, ravens and jays, are vulnerable 
to cache theft. These species use strategies that require 
self-control to minimise the chance of conspecifics 
stealing their caches, such as waiting for the optimal 
moment to make a cache, ie: when a competitor is out of 
Sight or out of earshot" (p2). 

Schnell et al (2022) tested ten Eurasian jays in two 
experiments. The first experiment measured various 
cognitive abilities, including spatial memory. In the 
second experiment the birds were taught that different 
drawers containing food would be pushed open by an 

xperimenter. Delay of gratification was tested by 
pushing open first the drawer with less preferred food 
and then it was closed, and the second drawer containing 
more preferred food was opened. The time to wait was 
varied. The second drawer did not open if the food was 
taken from the first drawer. The ability to wait was 
found to positively correlate with scores on the 
cognitive tests in the first experiment. 


—: 


3.7. EVOLUTION OF TAP IN HUMANS 


The evolution of TAP in humans must mean that the 
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ability was evident in human ancestors. The only evidence 
here is archaeological (Langley and Suddendorf 2022). 
Redshaw and Ganea (2022) summed up: "By around 1.8 Ma 


. 


[million years ago], for instance, Homo erectus 
individuals were manufacturing stone tools and 
transporting raw materials across vast distances, 
presumably because they recognised the possibility of 
using these tools and raw materials in the future. Later, 
around 500 000 years ago, Homo heidelbergensis 
individuals began assembling compound tools and 
apparently co-ordinating their actions in the service of 
shared goals — activities likely underpinned by an 
ability to reason through several interconnected steps in 
a possible causal chain. Only by around 100 000 years 
ago, however, is there compelling evidence of Homo 
sapiens and Neanderthals burying their dead, potentially 
reflecting sophisticated thoughts about the abstract 
possibility of an afterlife" (pp6-7). 


Imagining what is needed in the future leads to 
planning and preparation. Searching for evidence of these 
behaviours is the basis of archaeological research into 
TAP. 


Control of fire is one indicator of foresight. 
"Making fire requires considerable planning and 
persistence. Whether using firesticks or striking stones, 
resulting embers or sparks need to be carefully caught 
and fanned into a flame, requiring fire-feeding materials 
to be at the ready. Maintenance of fire also requires 
ongoing attention, particularly if certain temperatures 
are required for cooking or tool-making. While there ar 
glimmers of evidence for controlled use of fire reaching 
back some 1.6 Ma, habitual use of fire (systematically 
repeated use of fire in specific sites and/or regions) is 
only evident from about 400 000 to 300 000 years ago" 
(Langley and Suddendorf 2022 p2). 

Analysis of stone technology is another area of 
research. For example, the amount of time involved in 
making such tools requires the ability to imagine their 
future usefulness. Early evidence of foresight here 1.8 
Ma, including the movement of materials across landscapes 
in preparation (Langley and Suddendorf 2022). 

Hunting, particularly dangerous or difficult-to- 
catch prey, shows planning. "For example, when hunting 
large and/or dangerous animals, modern hunter-gatherers 
typically first observe the animal to learn its 
behavioural patterns (such as what path it takes to 
waterholes or rivers), before they plan the attack and 
prepare accordingly (which might include prepping 


f. 
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weapons, recruiting other hunters, digging a pit-fall or 
building a corral)" (Langley and Suddendorf 2022 p4). 
Direct evidence of technology use here around 500 000 
years ago (Langley and Suddendorf 2022). 

A variety of other technologies and artefacts signal 
thought about the future, including watercraft, semi- 
permanent forms of ornaments, and maps or hunting plans 
engraved on cave walls (Langley and Suddendorf 2022). 

Langley and Suddendorf (2022) ended: "Despite the 
vagaries of the archaeological record, it appears that 
the origins for the human ability for thinking about 
future possibilities stretches back beyond 1.8 Ma. From 
this early period of humanity, up through an increasingly 
complex family tree, the ability to conceive of 
the future and act to mitigate its risks or maximise a 
potential outcome developed gradually. These abilities 
cumulated in both individuals and groups co-operating on 
an ever-increasing scale and co-opting in increasing 
types of material culture to manipulate and track the 
future. Peopl ventually exchanged ideas about potential 
Situations, driving them on to avoid some possibilities 
while striving to turn others into reality" p8). 


3.8. FUTURE SELF 


Parfit (1971) suggested that "the sense of 
"‘connection' that individuals feel to distant, future 
selves may feel weak, and substantially weaker with 
greater periods of time, to the point that some very 
distant future selves may feel like different peopl 
altogether — and even, strangers" (Hershfield 2019 p72). 
But Hershfield (2023) argued that "people who feel 
close to their future selves - and realise they may be 
different to their present selves - make better 
decisions" (Frankel 2023 p46). 

There is support for the perception of the future 
self as similar to another person. For example, 
undergraduates were asked to make decisions for the 
future self, the current self, and a classmate (Pronin et 
al 2008). The participants "made decisions for their 
future selves that were more on par with the decisions 
that they made for their classmates rather than for 


themselves (eg: by signing their future selves up for a 
Similar amount of volunteer work as they assigned to 
others)" (Hershfield 2019 p72). 


Feeling a greater connection to the future self 
influences decisions made now (eg: Save more in a 
financial game; Bartels and Rips 2010). 
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Combining both a sense of connection and viewing the 
future self as another person has been studied (eg: Bryan 
and Hershfield 2012). University employees were asked to 
think about saving for retirement, but the message was 
presented in different ways - as a responsibility to the 
future self (self as other person condition), or in their 
best self-interest (self as same person condition). The 
former framing produced more changes to actual saving 
behaviour (Hershfield 2019). 

But Zimbardo and Boyd (2008) suggested that being 
too focused on the future leads to less willingness to 
help others (Frankel 2023). 


Fans 


3.9. APPENDIX 3A - PLACE CELLS 


The original work on rats and recording of 
electrical activity as they moved around, which led to 
the discovery of “place cells” was performed by O’ Keefe 
and Dostrovsky (1971). Subsequently, work with rats found 
other cells that responded to specific spatial positions 
(eg: “grid cells”; detailed in Moser et al 2008). Doeller 
et al (2010) reported evidence of such cells active in 
humans “moving” in virtual mazes. 

The idea of specific cells firing has been expanded 
to include “time cells” (details in Eichenbaum 2014), 
which fire as a means to measure the length of a task 
(but not “in a simple clock-like fashion” (p57); Schafer 
and Schiller 2023). “Some time cells encode space as 
well. In the brain, in fact, physical and temporal space 
may be bound together” (Schafer and Schiller 2023 p57). 

Grid-like patterns of activation of cells in the 
entorhinal cortex have been found for abstract 
associations (eg: two images) (eg: Constantinescu et al 
2016). This idea has been applied to the social world - 
“people maps” (Schafer and Schiller 2023). 

“That the same mapping system may underlie 
navigation through space and time, reasoning, memory, 
imagination and even social dynamics suggests that our 
ability to construct models of the world might be what 
makes us such adaptive learners. The world is full of 
both physical and abstract relations. Road maps of city 
streets and mental maps of inter-related concepts help us 
make sense of the world by extracting, organising and 
storing related information” (Schafer and Schiller 2023 


p59). 
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3.10. APPENDIX 3B - EPSTUDE ET AL (2022) 


Epstude et al (2022) observed that TAP provides 
"fertile ground for partisan reasoning. Counter-factual 
events, by definition, did not occur and hence cannot be 
verified. People may imagine counter-factual events that 
fit with their beliefs and motivations, asserting it 
‘almost happened' without the risk of being proved wrong. 
We propose that, when people reflect counter-factually 
about political events, political partisanship predicts 
what they imagine as well as what they infer — in other 
words, both the content and the conclusions of their 
counter-factual thinking" (pl). 

In their studies Epstude et al (2022) tested the 
main hypothesis that "partisans flexibly accept and 
generat ither upward or downward counter-factuals that 
are consistent with the preferred ideological stance or 
inconsistent with the opposed ideological stance" (p2). 
In this context, "upward" refers to more negative 
counter-factuals than actually happened and "downward" 
more negative. 

In the first two studies (Study la and 1b), 201 and 
192 US adults were recruited online with half of each 
group self-identifying as Democrat and half as 
Republican. They were presented with counter-factual 
statements related to the recent presidency, like: "If 
Trump had not passed the tax cuts, then the economy would 
currently be much better [worse]" (two versions of each 
statement). The response options varied from "extremely 
plausible" (5) to "extremely implausible" (1). Overall, a 
counter-factual statement was rated as more plausible 
when it aligned with the political view than misaligned 
(figure 3.1). 

Study 2, with 190 more participants, used other 
recent US political events - eg: "If Senate Republicans 
hadn't blocked Obama's appointee for the Supreme 
Court...", with the options "then things would have been 
better" or "then things would have been worse". The 
choice of a positive or negative counter-factual option 
was more likely to fit with the political view. 

These studies supported the main hypothesis, and 
showed that both political partisans were prone to 
cognitive bias in TAP. 
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Figure 3.1 - Mean plausibility score (out of 5) in Study 
la and lb. 
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4. ROUTE NAVIGATION 


Learning and recall are enhanced with salient 
stimuli. For example, on route-learning tasks, a 
brightly-coloured landmark aids correct recall of the 
direction at a junction (Davis et al 2017). 

Caduff and Timpf (2008) proposed three factors in 
the salience of a landmark - visual (eg: colour; size), 
structural (eg: proximity to a decision point), and 
semantic salience (ie: "salience due to personal, 
cultural, or historical meaning"; Redhead et al 2023 
pé6é18). 

Redhead et al (2023) focused on nostalgia as an 
example of semantic salient stimuli in their two 
experiments. Specifically, fond and personally meaningful 
memories of childhood and/or close relationships. 


Experiment 1 - Sixty participants in their 20s 
learned a route through a virtual maze. The landmarks at 
decision points in the maze showed pictures of popular 
music artists and TV characters, either popular during 
the participants' youth (nostalgia condition) or recent 
pictures (control condition). For example, a picture of 
the "Dr Who" actor 10 years ago (Matt Smith) or the 
recent actor (Jodie Whittaker). The time to complete the 
virtual maze was significantly faster in the nostalgia 
than control condition. 


Experiment 2 - This experiment used the same basic 
design with two differences - the landmarks were at 
decision or non-decision points along the route, and the 
landmarks were presented during learning but removed in 
the test trial. The participants were 48 high-school 
students. 

Nostalgia landmarks improved learning when placed at 
decision and non-decision points, and when used in 
learning and subsequently removed in the test, compared 
to control landmarks. 

So, in summary, "compared to more recent but equally 
recognisable landmarks, nostalgic landmarks enhanced 
spatial performance on a route learning task" (Redhead et 
al 2023 p626). 


x as 


The researchers commented on the application o 
their findings: "Navigating unfamiliar spaces can be 


challenging..., particularly within large-scale, multi- 
level indoor environments including libraries, museums, 
shopping centres, and hospitals... More often than not, 


these environments present long corridors and repetitive 
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pay 


designs, making them confusing and disorientating to 
navigate. Infusing landmarks with nostalgia could help 
address these design pitfalls by providing a meaningful 
visual aid" (Redhead et al 2023 p627). This will be even 
more so for individuals with neurological conditions like 
Alzheimer's disease. 
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5. MISCELLANEOUS 


5.1. Multi-talker situation 
5.2. Psychological cleansing 


5.1. MULTI-TALKER SITUATIONS 


Loud and quiet voices talking simultaneously are 
processed differently. The louder voice produced activity 
in the auditory cortex whether an individual was focused 
upon it or not, while the quieter voice required 
attention to be processed (Murugesu 2023). Raghavan et al 
(2023) measured the brain activity of seven individuals 
conscious while undergoing surgery for epilepsy. They 
listened to stories read by a male and a female speaker, 
initially individually (control condition), then two 
talkers at once with one designated as the target and the 
other as the non-target. 

It is easier when the target talker is louder than 


the background voices (known as "glimpsed target") than 
when the target talker is quieter (known as "masked 
target" or "glimpsed non-target") (Raghavan et al 2023). 
References 
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and masked speech in multi-talker situations PLoS Biology 21, 6, 
e3002128 (Freely available at 
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5.2. PSYCHOLOGICAL CLEANSING 


Cleansing behaviour includes personal hygiene, like 
washing hands, and brushing hair, and household tasks 
(eg: washing the dishes; vacuuming the floor). "Although 
norms regarding the form and frequency of cleansing vary 
between societies and historical periods..., the 
existence of hygienic care is a human universal" (Lee and 
Schwarz 2021 pl). 

Lee and Schwarz (2021) explored the psychology 
cleansing, beyond the obvious hygiene benefits of 
handwashing, for instance. For example, Xu et al (2012) 
found that, in a gambling game, participants who lost 
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continually reduced their subsequent bets (and the 
opposite for winning continually). But, if in the middle 
of the game, participants washed their hands (under the 
pretence of testing a new soap product), their bets 
returned to the original level. It was as if the losing 
or winning streak was washed away. This is the first type 
of "cleansing effects" (according to Lee and Schwarz 
2021) - ie: "effects of cleansing-related manipulations 
on psychological outcomes" (p2). 

The second type is “effects of psychological 
manipulations on cleansing-related outcomes" (Lee and 
Schwarz 2021 p2). Individuals ostracised in a co- 
operative computer game, for instance, were more likely 
to purchase cleansing (but non-cleansing) products (Poon 
2019). 

To explain these behaviours, Lee and Schwarz (2021) 
proposed a "grounded theory", where "mental processes 
(eg: knowledge, language, thought) do not reside ina 
layer of amodal symbols abstracted and detached from 
sensori-motor capacities for perception and action. 
Instead, the mental is grounded in the physical... 
Activating one activates the other" (p5). 
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6. SWEARING 


Swearing is “the use of specific, negatively charged 
and often emotionally loaded terms, which are taboo in a 
given language/culture and thus have strong potential to 
cause offence... It can be described as the use of words 
that ‘have the potential to be offensive, inappropriate, 


objectionable, or unacceptable in any given social 


context’ (Beers Fagersten 2012...)” (Stapleton et al 2022 
ppl-2). 

Singleton (2009) described three elements of swear 
words they relate to taboo subjects, have literal and 


non-literal meanings, and their use can be cathartic. The 
main taboo subject areas include religion (eg: “damn”), 
sex and sexual body parts (eg: “cunt”), and bodily 
excretions (eg: “shit”) (Stapleton et al 2022). 

“With reference to these categories, Hughes 
(1998...) notes that ‘swearing shows a curious 
convergence of the high and the low, the sacred and the 
profane’. Of course, even within these categories, there 
is some conflation, for example, between sexual versus 
excretory functions and body parts. There is also 
variation in the offensiveness of some of these core 
words/categories over time. Thus, the religious category, 
while still containing recognisable swear words, is 
likely to be less offensive to many people in 
contemporary society than it might have been in the past, 
and/or to constitute, today, a ‘milder’ form of swearing 
than many of the words in the sexual/excretory 
categories. By contrast, the ‘’’lower’’ physical 
faculties of copulation, defecation and urination’ 
(Hughes 1998...) have become much more prominent as swear 
word referents” (Stapleton et al 2022 p2). 


Stapleton et al (2022) outlined the issues related 
to the power of swearing: 


1. Emotional experience including catharsis and 
arousal - “Emotion is consistently cited in motivations 
for and perceptions of swearing, on both a cathartic and 
an expressive level; swearing is perceived by speakers to 
carry more emotional force than other forms of language; 
and empirical research confirms a strong link between 
swearing and emotional arousal” (Stapleton et al 2022 
pp3-4). 

Swearing in the first language is perceived as more 
emotionally intense than in languages acquired later in 


life by multi-lingual speakers (Stapleton et al 2022). 
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2. Attention and memory - Swear words gain more 
attention and produce stronger memory recall than non- 
taboo words in experiments. 

For example, Jay et al (2008) (Experiment 1) 
presented students with 36 words individually taken from 
thr categories taboo (eg: “cock”), neutral (eg: 
“brother”), and emotional non-taboo (eg: “cuddle”), and 
asked them questions like “Is the word in capital 
letters?”. Later there was a memory tests for the words, 


a a 


and average recall was 39% for taboo words compared to 


Q 


emotional (13%) and neutral (7%) words. 


3. Physiology - Swear words produce increased 
autonomic nervous system activity compared to neutral or 
non-taboo words in experiments (eg: Jay et al 2008: 
Experiment 2). 


4. Pain tolerance and relief Stephens et al 
(2009), for example, found that participants could hold 
their arm in ice water for longer when allowed to swear 
than use non-Sswear words (mean 190 vs 146 seconds). 


5. Interpersonal functions - The distinction has 
been made between “annoyance swearing” and “social 
Swearing” (Ross 1960). Concentrating on the latter type, 


Swearing can be used directly as an insult or to 
intensify an insult, or to offend (ie: negative 
functions). While swearing can be used to enhance 
relationships in a group (ie: positive functions). 

For example, Beer Fagersten (2017) found benefits 
for YouTuber’s in building para-social relationships with 
followers, while hypothetical Italian local political 
candidates who used swear words in fictitious blogs (eg: 
“is up shit creek”) received more positive evaluations 
than non-swearers (eg: “is in a tragic situation”) (but 
not necessarily likelihood to receive vote rating). 
Gender was also important, and female swearers did 
poorer. (Cavazza and Guidetti 2014). 


In summary: “Swearing produces effects that are not 
observed with other forms of language use. Thus, swearing 
is powerful. It generates a range of distinctive 
outcomes: physiological, cognitive, emotional, pain- 
relieving, interactional and rhetorical” (Stapleton et al 
2022 pl). 
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7. CONCEPTS AND SOCIAL INTERACTION 


7.1. Introduction 
7.2. Defining concepts 


7.2.1. Abstract concepts 
7.3. Concepts emerging during social interactions 
7.3.1. During development 


7.3.2. During dialogue 
7.3.3. In the brain 
7.4. Concepts and interactions with ourselves 
7.4.1. In sensori-motor and inner experienc 
7.4.2. Inner speech (and language and thought) 
7.5. References 


7.1. INTRODUCTION 


“Concepts represent the building blocks of 
knowledge; they are crucial for thinking, inferring and 
interacting with the environment. They are the ‘glue’ 
that connects our past, present and futur xperienc 
Concepts are typically investigated in two ways: in their 
relation to categorisation — they can be considered as 
the cognitive and mental aspects of categories — and as 
instruments of thought” (Borghi et al 2022 pl). The key 
questions are what are concepts, their formation, and 
their relationship to language (Borghi et al 2022) 1'. 

But concepts do not exist in a vacuum, they arise 
from social interactions, or in “situated interactions”, 
to be more specific (Borghi et al 2022). “Studies show 
that interaction facilitates abstract thought and 
problem-solving and reveal that emotions and social 
interaction are paramount for abstract concept 
acquisition and use” (Borghi et al 2022 p2). 

There is another issue, Borghi et al (2022) 
explained, of “how different ways of interacting with 
ourselves potentiate our concepts and cognition. Language 
is a powerful instrument that enriches our cognitive 
abilities. Work on inner speech has recently had a novel 
impulse, showing that we use different kinds of inner 
speech — monologic and dialogic, condensed and expanded — 
and that inner speech influences and enhances our 
thinking processes” (p2). 


') The traditional view of concepts is that they “derive their semantic content by being grounded in 
perception, action and introspection, with ‘grounding’ being defined as the process through which 
representations are connected to what they are about, the capacity to autonomously link them with 
referents in the outside world” (Mannella and Tummolini 2022 p2). An alternative to this “grounded 
cognition” view is that concepts emerge from an intrinsic motivation of competence acquisition 
(Mannella and Tummolini 2022). 
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Introducing a special issue of “Philosophical 
Transactions of the Royal Society B”, Borghi et al (2022) 
outlined the following key areas of interest for concepts 
and social interactions: 


i) Defining concepts. 


ii) Concepts emerging from social interactions: 


a) During development. 
b) During dialogue. 
c) In the brain. 


iii) Concepts and interaction with ourselves: 


a) In sensori-motor and inner experiences. 
b) Inner speech. 


7.2. DEFINING CONCEPTS 


Shea (2022) saw concepts as “the recombinable 
components of conscious, deliberate thought” (pl). They 
are “found in our judgements, hypotheses, intentions, 
aims and occurrent desires and beliefs. Their subject 
matter encompasses the concrete and abstract, particulars 
and properties, the physical, logical and socio-cultural, 
and runs from the mundane to the extraordinary: DOG, 
NUMBER, HEAVY, AND, MONDAY, BUS, MANDELA.1 They are sub- 
propositional: a concept does not make a claim about the 
world on its own, but only when combined with other 
concepts into a complete thought” (Shea 2022 pl). 

Traditionally research has focused on the 
categorisation of concepts, but Shea (2022) argued that 
“Just as important is the capacity to use concepts to 
think through what to do, or to work out what is the 
case, unprompted by a current incoming stimulus. These 
episodes of thinking begin with a conceptual thought” 
(pl). Each concept provides access to information, 
whether it be stored memories or running “simulations” of 
what could be. Simulation is particularly important in 
reasoning (eg: “if I do X, then Y will follow, but if I 
do X1, then Y will not follow”). A concept is a “plug and 
play device”, argued Shea (2022), that allows the 
“special-purpose systems” of cognition to run the 
relevant simulation (eg: planning a route; making a 
particular comment to a person). 

Shea (2022) offered the example of the thought of 
“soldiers mounted on polar bears”; “you will start to 
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imagine what that will look like and what might happen. 
There is considerabl vidence that thinking with 
concepts activates representations in multiple special- 
purpose systems — sensory, motoric, affective, evaluative 
and amodal. The power of conceptual recombination lies 
not simply in the capacity to recombine labels or words 
in a language of thought. Its power lies in the capacity 
to simulate scenarios corresponding to our thoughts” 

(p4) . 


Concentrating on linguistic concepts, Enfield (2022) 
distinguished between the “O-axis” and “I-axis”. The O- 
axis (O = objective) focuses on the sign standing f 


for an 
object, “where the sign is something we can directly 
perceive, such as the sound of a word, and the object is 
a concept, or the sense of a word, often implying 


reference to a corresponding class of physical entities 
(a category) in the world” (Enfield 2022 pl). The I-axis 


(I = interpretation) describes the link between concepts 
and social interactions. “On the O-axis we can ask ‘What 
is the definition of scalpel?’, while on the I-axis we 


can ask how people respond to the word. What does the 
word give rise to in the flow of social interaction? So, 
we might hear a surgeon utter the word scalpel and see 
their assistant pass a scalpel in response” (Enfield 2022 
pl). 


Enfield (2022) emphasised the need for both axes to 
ully understand linguistic concepts: “Without social 
functions, the signs that denote linguistic concepts 
would not circulate, and the conventionalised concepts 
they denote would not be reproduced and would not exist. 
Linguistic concepts are not thoughts that happen to be 
made public. Linguistic concepts are social events that 
happen to become thoughts” (p6). 


ma 


7.2.1. Abstract Concepts 


Langland-Hassan and Davis (2022) were interested in 
abstract concepts, which they considered based around two 
central questions: 


i) What makes a concept abstract, and what makes one 
concept more abstract than another one? 


ii) What is the relationship between abstract 
concepts and language? 


Concerning the first question, “Borghi et al [2017] 
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describe abstract concepts (such as FANTASY, FREEDOM and 
JUSTICE) as ‘lacking bounded and clearly perceivable 
referents’ and as being ‘more detached from sensorial 
experience than concrete ones’... Borghi et al [2018] 
also note that abstract concepts ‘do not possess a single 
and perceptually bounded object as referent’, and have a 
content that is ‘more variable both within and across 
individuals’” (Langland-Hassan and Davis 2022 p2). 
Barsalou (2003) emphasised that abstract concepts are 
“detached from physical entities”, and are “more 
associated with mental events” (quoted in Langland-Hassan 
and Davis 2022). 

Researchers have tended to study abstract concepts 
using “concreteness ratings”. Brysbaert et al (2014) 
explained: “Some words refer to things or actions in 
reality, which you can experience directly through one of 
the five senses. We call these words concrete words. 
Other words refer to meanings that cannot b xperienced 
directly but which we know because the meanings can be 
defined by other words. These are abstract words. Still 
other words fall in-between the two extremes, because w 
can experience them to some extent and in addition we 
rely on language to understand them. We want you to 
indicate how concrete the meaning of each word is for you 
by using a 5-point rating scale going from abstract to 
concrete” (quoted in Langland-Hassan and Davis 2022). 

A similar notion of concepts is “abstractness-as- 
imperceptibility”. Put simply, whether the concept can be 
perceived by the senses. That seems very much like 
concreteness as defined above, but Langland-Hassan and 
Davis (2022) saw concreteness and imperceptibility as 
interacting to produce four classes of abstract concepts: 


a) “Minimally abstracted concepts of perceptibles” - 
These concepts can be perceived by the senses, and show 
perceptual similarities to other concepts (eg: cow, 
table, cup). 


b) “Highly abstracted concepts of perceptibles” - 
Similar to (a), but the concepts have relatively few 
Similarities to other concepts (eg: object, organism, 
plant). 


c) “Minimally abstracted concepts of imperceptibles” 
- Imperceptible to the senses, but share similarities 
(eg: seven, lectron). 


d) Highly abstracted concepts of imperceptibles” - 
Imperceptible with few similarities (eg: value, mental 
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state). 


Langland-Hassan and Davis (2022) wondered whether 
categorisation of these different types of concepts 
varies. The categorisation of concepts has been studied 
with the “Cactus to Camel test”, where a participant is 
presented with a target image and must link it to one of 
four other images. For example, a target image of a 
passenger train, and the images of a motorcycle, a 
tractor, a small truck, and a bus. The choice of bus 
suggests a single category of “mass-transit vehicles” 
(Langland-Hassan and Davis 2022). 

Another example might be a target object of electric 
socket plug, and a choice of a candle, flashlight, 
lantern and lightbulb. Matching the plug with the 
lightbulb suggests the concept “electricity” (Langland- 
Hassan and Davis 2022). 

Langland-Hassan et al (2021) used this test, and 
after the choice of matching image, participants were 
asked to give one word that linked the two images. The 
linking word was then rated for its “concept 
concreteness”. The linking word and its concreteness 
could vary depending on the context (ie: the choices), 
and lLangland-Hassan and Davis (2022) called this “trial 
concreteness”. The linking word varies in concreteness 
depending on the visual similarity of the choices, for 
instance, even if the same word is used. 

For example, both linking words in table 7.1 are 
“dog”, but the trial concreteness is different because of 
the visual similarity differences in the two trials. In 
trial (a), the non-matching choices are very different to 
those in trial (b). 


Target Image|Choice Images Answer Linking Word 
German Fork, Knife, Spoon, Poodle Dog 

shepherd Poodle 

German Hyena, Tiger, Lion, Chihuahua Dog 

shepherd Chihuahua 


(Source: Langland-Hassan and Davis 2022 figure 1) 


Table 7.1 - Two examples of the Cactus to Camel test. 


“ 


Langland-Hassan and Davis (2022) asserted that 
see minimal value in assigning a fixed degree of 
abstractness to any individual concept, given that this 
value will itself be a kind of averaging across very many 


we 
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contexts where varying degrees of abstraction were 
required for the behaviours we associate with use of the 
concept. It remains true that some concepts will, on 
average, require more abstraction for their use than 
others. In that sense, which averages across all usages, 
we can say that one concept is more abstract (in tending 
to require, for its use, more abstraction away from 
differences) than another. Concepts of more inclusive 
categories (such as MAMMAL) will tend to require more 
abstraction than narrower categories (such as DOG). 
Nevertheless, depending on the context, these differences 
may be reversed on any particular task that aims to test 
facility with a concept. If we wish to investigate the 
support language may provide for abstract conceptual 
abilities, it is more useful to rate the degree of 
abstraction required by a particular use of a concept in 
context” (pp9-10). 


A distinction is usually made between concepts 
concerning objects (entity categories) and concepts 
concerning relations (relational categories) '*. The 
former are directly perceived and the latter subsequently 
inferred in one view, while an alternative perspective is 
that “relations are perceived directly, as they are 
part of the anticipatory organisation of the cognitive 
system that were stabilised while acting in a given 
environment” (Raczaszek-Leonardi and Zubek 2022 p3). This 
perspective can be called “action-first” or “interaction- 
based” (Raczaszek-Leonardi and Zubek 2022). 

“Affordances” is an important idea here. What is 
directly perceived are “relational properties of an 
organism in its environment, which can guide possible 
actions. Instead of perceiving the objective shape and 
size of a chair, an agent notices that the shape and size 
are just right to offer the possibility of sitting” 
(Raczaszek-Leonardi and Zubek 2022 p3). 


Finding similarities and differences between 
concepts from the same semantic categories “is easy - at 
least for adults” (Liu and Lupyan 2022 pl). For example, 
a doctor and a nurse are judged as more similar than a 
doctor and a carpenter. 

“However, people’s ability to appreciate semantic 


'? Twardowski (1924 quoted in Raczaszek-Leonardi and Zubek 2022) called relational concepts 
examples of “analytical concepts”, and entity concepts “synthetic concepts”. The distinction is based 
around the “imaginability” of a concept. “Analytical concepts are constructed out of percepts or other 
concepts by selecting some parts of the perceived whole that cannot be imagined on their own, ie:. 
without entities that have these properties or exist within certain relations (for example, having a 
colour, being long, or being a daughter). By contrast, entity categories are Twardowski’s synthetic 
concepts, as they are characterised by ‘wholeness’” (Raczaszek-Leonardi and Zubek 2022 p7). 
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Similarity is not limited to comparing items from the 
same semantic domain. When asked to align concepts from 
different semantic domains, people sometimes show uncanny 
convergence. For example, when asked ‘If science were a 
colour, what colour would it be’, 40% respond with 
‘green’. When asked in the same manner, to map philosophy 
to a beverage, 20% map it to tea. When mapping 
professions to musical instruments, 32% map doctors to 
pianos while 26% map nurses to violins [Liu and Lupyan 
2020 quoted in Liu and Lupyan 2022] (these two responses 
are the modal responses and greatly exceed the 
probability of these instruments being mentioned by 
chance alone)” (Liu and Lupyan 2022 pl). 

Liu and Lupyan (2022) performed further experiments 
on cross-domain mappings of concepts. Participants were 
asked, “If a nurse were an animal, they would be a(n)”, 
and 20% answered “cat”. Liu and Lupyan (2022) ended: “Our 
results show that people tend to converge in how they map 
between concrete domains and that they do so either by 
making use of abstract information encoded as part of the 
concrete concepts that are being mapped, or by actively 
projecting these concepts into a more abstract semantic 
space so that they can be aligned” (p13). 

Such studies suggest that concrete and abstract 
concepts are stored based on dimensions like valence. “It 
is reasonable to talk about positively and negatively 
valenced animals, jobs, beverages, colours etc. and one 
can imagine that mapping between these domains is 
informed by how similarly valenced the items are. For 
example, if part of our representation of ‘rat’ is its 
negative valence, then when we are asked ‘If a rat were a 
job, what job would it be’, we tend to think of 
negatively valenced jobs” (Liu and Lupyan 2022 p2). 


7.3. CONCEPTS EMERGING DURING SOCIAL INTERACTIONS 


7.3.1. During Development 


The study of the brain during learning has tended to 
be a “single-brain” approach (Gazzaniga et al 2002) (ie: 
a focus on how the individual learns), but more recently 
a “second-person neuroscience” approach (Redcay and 
Shillbach 2019) has emerged. This approach “studies 
cognitive processes in interaction, including the back- 
and-forth dynamics between two or more people” (De Felice 
et al 2022 pl). 

De Felice et al (2022) argued for this approach 
suggesting that “(solitary) action per se may not be 
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enough, but rather that action in interaction may be key 
to support human learning, as exercised via many forms 
including gestures, object manipulation and language” 
(p2). This interactive learning (ie: learning with 
others) is distinct from learning from others as in 
observational learning and imitation (De Felice et al 
BOZLY 5 

The few studies of interactive learning mostly 
concern language acquisition. For example, Kuhl et al 
(2003) taught nine month-old “English-speaking” babies to 
distinguish Chinese Mandarin sounds in one of three 
conditions - in interaction with a native speaker, via 
video, or audio recording. “Despite equivalent 
exposure time and content of Chinese sounds, only the 
group who engaged in live-interaction with the teacher 
showed learning, and being exposed to videos or sound 
recordings was associated with no learning. While this 
study is not strictly looking at knowledge-based learning 
(eg: concepts), it provides strong evidence for the 
crucial role of interaction in children’s learning over 
non-interactive learning methods” (De Felice et al 2022 
DS). 


ve 


Similar research has shown the benefits of 
interactive learning for adults. Jeong et al (2021), for 
example, taught Japanese-speaking adults Korean spoken 
words via videos depicting social interactions, or in the 
translation of the words. Learning was better in the 
first condition. 

As mentioned above, there is little research on 
interactive learning of concepts. De Felice et al (2022) 
speculated: “Interactive teaching would allow a learner 
to try out a new concept and explore how it relates to 
other concepts with immediate feedback, which is likely 
to provide richer and more robust learning” (p8). 

In terms of what is happening in interactive 
learning, De Felice et al (2022) admitted that it is 
difficult “to disentangle the contribution of specific 
factors associated with social contexts that benefit 
human learning. It is hard to separate individual 
components because live interaction cannot be easily 
deconstructed. Future studies using virtual reality might 
be able to do so by experimentally manipulating which 
aspects of interaction are most important to learning” 
(ps). 


In another study of interaction in learning, 
Karmazyn-Raz and Smith (2022) observed thirty-two 
toddler-parent interactions for ten minutes in a playroom 
with a choice of 32 toys. The choice of toys was not 
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random, but showed a pattern that provided a scaffolding 
for learning. “Episodes of toy play may b xperienced 
and remembered, as are stories, as a coherent system of 
meaningful relations” (Karmazyn-Raz and Smith 2022 pll). 


Viertel et al (2022) studied the acquisition of 
abstract concepts specifically in the form of religious 
words like mercy, blessing, and holy. “Between the ages 
of 8 and 9 years old, children achieve the largest 
learning progress in acquiring the meaning of abstract 
words [...] In contrast with concrete words, the 
challenge in learning abstract words is to systematically 
connect the elements that constitute a concept, which are 
dislocated across space and time” (Viertel et al 2022 


p2). 


Three processes are involved in the acquisition of 
abstract concepts (Viertel et al 2022): 


i) Emotionalisation - The association of an 
emotional experience with the word as way to anchor the 
novel meaning. 


ii) Perspectivation - “For example, when reading a 
story aloud, the caregiver could take on the perspectiv 
of a character and convey an emotion from that 
character’s perspective, ie: she or he focuses on a 
specific mental or emotional state” (Viertel et al 2022 


pays: 


iii) Interactionist contextualisation - A concept is 
co-constructed in social interactions. 


Viertel et al (2022) explored these processes in a 
pilot study with eight 7-8 year-olds and caregivers in 
Germany and the word “mercy”. Firstly, the caregiver read 
the Bible story of Jonah to the child and the concept of 
mercy was discussed. Then the child was presented with 
pictures, some linked to the story, and asked to choose 
those showing mercy. 

There was evidence of emotionalisation and 
perspectivation being used. For example, the latter was 
seen in the caregiver encouraging the child to perceive 
another person’s feelings or imagine their motives in the 
reading session. The children, however, were not able to 
generalise mercy to secular stories, and instead used 
related concepts like kind. 

Viertel et al (2022) explained their thinking: “We 
chose the word mercy as an object of investigation 
because it is a prototypical religious word which anchors 
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emotionalisation and perspectivation through language. 
Our starting point was that the mere naming of 
semantically relevant aspects is not sufficient to 
experience the profound meaning of it. Instead, an 
enrichment by the related linguistic means is necessary 
but has to be constructed by child and caregiver 


together” (p7). 


7.3.2. During Dialogue 


“Dialogues are everyday co-operative joint 
activities, involving more than one person who seeks to 
comprehend each other. Given how common it is and how 
natural it feels to take part in dialogues (from ordering 
a coffee ina bar to chatting with friends over a meal), 
one might risk overlooking the cognitive challenges that 
dialogue offers to speakers” (Gandolfi et al 2022 pl). 
Mutual understanding here involves alignment ie: when 
the speakers “focus on the same aspects of the world and 
conceptualise them in the same way” (Gandolfi et al 2022 
pl). 


But this process is a collaboration, and 
“interlocutors share control control over the dialogue by 
continuously monitoring and comparing their own and their 
interlocutor’s contributions, by drawing on metacognition 
and social cognition, and specifically by meta- 
representing whether they believe they and their 
interlocutor are aligned or not. This process allows 
speakers to build similar representations and align on 
their way of understanding complex phenomena in the world 
without having to build elaborate inferential systems of 
beliefs about their own and their interlocutor’s 
understanding” (Gandolfi et al 2022 ppl-2). Alignment has 
greater significance for abstract than concrete concepts. 

Pickering and Garrod (2021) explained that speakers 
collaborate in “the shared workspace” (ie: “the set of 
salient signs and associated context that are at both the 
speakers’ disposal”; Gandolfi et al 2022 p2). 


A similar process is the “division of linguistic 
labour” (DLL), which is the sharing of knowledge about a 
particular concept. Originally seen as novices dependent 
on experts for the identification of some concepts (eg: 
Putnam 1975), Andrade-Lotero et al (2022) broadened the 
framework “to capture situations in which the members of 
a community do not necessarily differ in their overall 
level of knowledge, but rather in their perspective or 


kind of knowledge” (pl). 
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Put another way, the goal of communication for 
individuals is to “make use of cognitive specialisation 
to increase the efficiency of their own cognitive 
processes” (Andrade-Lotero et al 2022 p3), which is 
contrary to the “common ground” view (eg: Clark 1966) 
that characterises “communication as a process that 
requires a shared background of knowledge for the 
activity to be successful” (Andrade-Lotero et al 2022 
p3). The “pooled ground” approach (eg: Raczaszek-Leonardi 
et al 2014) is another way of describing the DLL. 

Andrade-Lotero et al (2022) reported an experiment 
on the DLL. Highty-four students at a university in 
Colombia were recruited for the two conditions - solitary 
and paired. The task was to identify correctly the breed 
of dog shown (eg: “Is this dog a Cairn terrier?”) either 
working alone or with a partner. More accurate 
classification occurred in the paired condition, 
particularly when the players asked their partner for 
clarification of the breed. 


Olsen and Tylen (2022) argued that social 
interaction aids the forming of abstract representations 
(abstraction): “interaction can stimulate processes of 
abstraction when varied individual information is 
integrated and generalised across members of a group, 
facilitating the flexible transfer of experiences and 
knowledge between contexts” (p7). 

This idea has been tested in experiments involving 
solving problems working alone or in pairs. For example, 
Schwartz (1995) found that pairs produced more abstract 
representations of science problems which aided their 
solution than individuals alone. While Voiklis and Corter 
(2012) found similar results in a complex, rule-based 
categorisation task. 


“Interpersonal attunement” is a more general term 
that has been used. “People tend to ‘fall in synchrony’ 
in social interactions, eg: through spontaneously 
aligning rhythmic behaviours, such as gait and clapping 
or adopting others’ mannerisms, from shaking a f 


foot or 
scratching the head to adopting each other’s speech 
styles, emotions and moods [...] All of these instances 
of interpersonal attunement can be thought of as 
facilitating human communication, collaboration and 
eventually trust-based social relationships...” (Bolis et 
al 2022 ppl1-2). 

Misattunement can be seen as the basis of 
psychopathology (eg: “dialectical misattunement 
hypothesis”; Da Costa et al 2022). 
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7.3.3. In the Brain 


The social qualities of words (or socialness) has an 
impact on the structure of concepts and the 
representation of our semantic knowledge in the brain. 
This is particularly important for abstract words (Pexman 
et al 2022). 

For example, Troche et al (2014) asked participants 
to rate 200 concrete and 200 abstract English nouns on 
twelve dimensions, which factor analysis reduced to three 
underlying factors affective association/social 
cognition, perceptual salience, and magnitude. “Abstract 
word meanings relied more heavily on affective 
association/social cognition than did concrete meanings” 
(Pexman et al 2022 pp2-3). As similar idea of clusters 
was seen in research with Italian nouns (Villani et al 
2019), though the clusters were different - 
philosophical/spiritual concepts; physical, spatio- 
temporal and quantitative concepts; emotional/inner state 
concepts; and self and sociality concepts, while Vargas 
and Just (2020) found patterns in neural signatures as 
participants thought of 28 abstract words in a functional 
magnetic resonance imaging scanner (3 latent factors - 
verbal representation, externality/internality, and 
social content) (Pexman et al 2022). 

Concentrating on the brain and socialness, there are 
two main theories (Pexman et al 2022): 


i) The “social knowledge hypothesis” (eg: Olson et 
al 2013) - Multiple independent stores in the brain 
compose the semantic system, and the dorso-lateral 
aspects of the anterior temporal lobe (ATL) appears to 
represent social knowledge (according to neuroimaging 
studies). 


ii) The “graded semantic hub” explanation (eg: 
Binney et al 2012) - This also focuses on the ATL, but 
“the whole ATL comprises a unified semantic 
representational space, all of which is engaged by the 
encoding and retrieval of concepts, and by concepts of 
any kind” (Pexman et al 2022 p4). 


Pexman et al (2022) noted that there is no agreed 
definition of “socialness” which hampers theories and 
research. For these researchers, socialness, “broadly 
construed, is a dimension of word meaning that can be 
distinguished from other dimensions such as concreteness 
and valence. Moreover, there is some evidence that 
socialness is reflected within the organisation of neural 
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systems that support semantic processing. It remains to 
be seen whether this is indicative of social words being 
a distinct type, or whether socialness is just one of 


many dimensions that define a unified domain-general 
semantic space” (Pexman et al 2022 pp5-6). 


Activity in the insula, the amygdala, and several 
cortical areas have been found for social concepts (Lopes 
da Cunha et al 2022). “Social concepts are those that 
primarily evoke socio-interactive behaviours, traits or 
circumstances. In the linguistic domain, these can be 
conveyed by verbs, nouns, adjectives and adverbs with 
positive or negative valence (such as help, kindness 
or friendly, and resent, envy or jealous)” (Lopes da 
Cunha et al 2022 pl). Also the cerebellum and its 
connectivity to the cortex is involved according to 
research by Lopes da Cunha et al (2022). 

Fifteen Spanish speaking cerebellar ataxia (CA) 
patients were compared to 29 healthy controls (HCs). CA 
is “a disorder typified by cerebellar atrophy and 
abnormal cerebello-cortical connectivity, as well as 
disruptions of socio-cognitive domains — such as emotion 
attribution, mental state recognition and theory of mind” 
(Lopes da Cunha et al 2022 p2). Comprehension of two 
types of passages read out to the participants was the 
outcome measure. The text was classed as social 
(involving interpersonal events - eg: “Immediately, he 
went over to Juan and earnestly asked for a favour”; p4) 
and non-social (a single person’s actions - eg: 
“Afterwards, he would read a book and listen to classical 
music on the balcony”; p4). There was no difference in 
comprehension scores for the non-social text, but the CA 
group had significantly lower scores for the social text. 


7.4. CONCEPTS AND INTERACTION WITH OURSELVES 
7.4.1. In Sensori-Motor and Inner Experiences 


One idea is that concrete concepts involve sensori- 
motor neurons in the brain (eg: the activation of leg- 
related neurons when thinking about the concept “kick”) 
(eg: Barsalou 2008), but this is not possible for 
abstract concepts. The solution is “emotional grounding” 
(or affective grounding” or “affective embodiment”) (eg: 
Lenci et al 2018). Abstract concepts activate emotion- 
related neurons in the brain (Winter 2022). 

The main line of evidence in favour of emotional 
grounding is the correlation between “emotional 
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experience” (defined as “the relative ease with which a 
concept evokes emotional experiences”; Winter 2022 p2) 
and the abstractness of a concept (eg: Newcombe et al 
2012). The studies, however, are mixed in their findings 
(Winter 2022). 

Winter (2022), using words in Cantonese, Mandarin 
Chinese, Croatian, Dutch, French, German, Indonesian, 
Italian, Polish, and Spanish, found that “the idea of 
emotional grounding only characterises a small sub-set of 
abstract concepts” (pl), and depending on how 
concreteness/abstractness is operationalised, “concrete 
concepts are rated as more emotional than abstract ones” 
(pl). 


Banks and Connell (2022) found a more complex 
picture in their three experiments. They found that “both 
concrete and abstract categories are strongly grounded in 
multi-dimensional sensori-motor experience. Both domains 
were dominated by vision and, to a lesser extent, head 
movements, but concrete categories were more grounded in 
touch and hand-arm action, while abstract categories were 
more grounded in hearing and interoception. Importantly, 
this pattern of grounding was not uniform, and sub- 
domains of concrete (eg: ingestibles, animates, natural 
categories and artefacts) and abstract (eg: internal, 
social and non-social) categories were grounded in 
different profiles of sensori-motor experience” (Banks 
and Connell 2022 pl). 

English-speaking participants were asked to rate 
concepts as to the extent each was experienced via six 
perceptual modalities (vision, hearing, taste, smell, 
touch, and interoception) and five body area actions 
(hand-arm, foot-leg, head, mouth, and torso). The eleven 
dimensions were combined to give each concept an overall 
score, where a higher score was a stronger sensori-motor 
grounding. 

The mean score for concrete concepts was slightly 
higher than for abstract concepts, but “both abstract and 
concrete categories had member concepts that were quite 
weakly (eg: nano-second as a unit of time, hydrogen as a 
chemical element) and strongly (eg: pain as a symptom of 
illness, shower as a bathroom fixture) grounded” (Banks 
and Connell 2022 p3). 


The mental mapping of emotional concepts is an area 
of interest. Two dimensions appear important - hedonic 
valence (eg: pleasure-displeasure), and level of 
physiological activation/arousal (Barca et al 2022). 
Barca et al (2022) argued, however, that “emotional 
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concepts are not processed in the void, but rather in the 
context of other emotional stimuli and more broadly of a 
variety of factors (eg: emotional dispositions, 
physiological or interoceptive states) that are known to 
impact emotion perception and representation” (p2). 

They supported this view with experimental evidence. 
Thirty adults were presented with 24 emotional words in 
sets of three, and had to choose two that were similar. 
The context (ie: the other words presented) influenced 
the choices, but more so for words perceived as low on 
the arousal dimension. 


£ 


The naming of smells involves olfactory concepts, 
which may be abstract (eg: citrusy smell”) or concrete 
(eg: “the smell of lemon”) (Deroy 2022). Deroy (2022) saw 


an over-representation of abstraction in relation to 
olfaction, which is contrary to other senses. She 
proposed two reasons, as Borghi et al (2022) explained: 
“The first is that abstract concepts make people feel 
less authoritative and defer more to others; hence there 
would be a benefit in terms of social cohesion. The 
second is that converging on abstract terms would be 
easier. Hence, among the adaptive advantages that the use 
of abstract terms can provide, she identifies the social 
feedback received by others, which could increase 
people’s confidence that their communicative intention 
has been understood. Both hypotheses strongly highlight 
the link between concepts, metacognition, and social 
interaction” (p5). 

Olfactory terms can be plotted on two dimensions - 
degree of abstractness, and specific-general. For 
example, “the smell of lemon” is low generalisation and 
abstractness, while “acrid smell” is high on both 
dimensions (Deroy 2022). 

Naming smells is difficult, and Jonsson and Olsson 
(2012), for example, found around 20% accuracy for common 
smells. Naming colours is much better (Deroy 2022). 
However, there are cultural differences. For example, the 
Jahai people (indigenous hunter-gatherers who live in 
rainforests in Malaysia and Thailand) have only 12-15 
general smell terms. “Under experimental conditions, the 
Jahai are as good at naming smells using these general 
words as they are at naming colours. They are clearly 
better at odour naming than matched English speakers 
[Majid and Burenhult 2014], as measured in response 
times” (Deroy 2022 p3). 

Naming common smells involves three stages, 
according to Oloffson and Gottfried (2015): object 
perception (ie: identifying a smell), lexical-semantic 


Psychology Miscellany No. 202; June 2024; ISSN: 1754-2200; Kevin Brewer 


78 


integration (ie: finding the name in the memory), and 
verbalisation (of the answer). There are potential 
problems at each stage. “When participants struggle to 
come up with a name for odourants during experiments, 
some of their difficulties start with not being able to 
perceptually identify a distinct smell, and/or to 
construct it as a distinct object for naming. Other 
limitations, besides difficulties in perceptual 
identification of single-molecule odourants, make 
olfaction arduous in the real world. Discriminating and 
identifying component smells in mixtures remains 
particularly challenging” (Deroy 2022 p4). 

While verbalisation problems include the wording 
used by the researchers for example, “What does it 
smell like?”. Deroy (2022) argued that such wording 


cay 


favours concrete responses. Poulton (2020), for example, 


used the wording, “How does it smell?”, and found 
differences to other studies (Deroy 2022). 


7.4.2. Inner Speech (and Language and Thought) 


The relationship between language and thought has 
interested researchers for much of the history of 
psychology. Simply, there are two views - that language 
determines thought, or language-independent concept 


= 


formation occurs. The former is best seen in the (Sapir- 


Ras 


Whorf) “linguistic relativity hypothesis” (eg: Whorf 
1956). For example, the words in a language for different 
colours influence the perception of different colours. 
More recently, the “label feedback hypothesis” proposed 
that labels “play an active role in perception and 
categorisation by selectively activating perceptual 
features that are diagnostic of the category being 
labelled” (Lupyan 2012 quoted in Henningsen-Schomers et 
al 2022). 

Using computational modelling, Henningsen-Schomers 
et al (2022) found partial support for the linguistic 
relativity hypothesis, namely that “‘Whorfian’ effects 
should be modulated by the concreteness/abstractness of 
the semantic categories being acquired, with language 
labels supporting the learning of abstract concepts more 
than that of concrete ones” (pl). 


Dove (2022) took this position: “Viewing language as 
a component of multi-modal embodied cognition suggests 
that the presence, or indeed the absence, of words should 
influence our reasoning. Words should enhance and 
transform our cognition in specific ways” (p4). He 
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proposed the following ways. 

Firstly, “words facilitate generalisation. They may 
serve as representational anchors that help stabilise and 
organise embodied simulations of experience. Words (but 
not tones, sounds or emotion responses) have been found 
to help young children with object individuation. Verbal 
cues such as the word dog appear to activate more general 
category representations than non-verbal cues such as the 
sound of a dog barking” (Dove 2022 p4). Secondly, words 
focus attention to specific conceptual features, and 
thirdly, “words may engage background frames or schemas. 
For instance, the meanings of the English words buy, 
cost, pay and sell are all understood relative to a 
commercial event f 


frame. A word like vegetarian makes 
sense in a society where many peopl at meat” (Dove 2022 
p4). 


The relationship between inner speech (“covert self- 
directed talk”; Borghi and Fernyhough 2022) and the 
acquisition and use of concepts is well studied, Borghi 
and Fernyhough (2022) saw it as more important for 
abstract concepts. The importance of inner speech in 
children’s development was particularly by Lev Vygotsky 
(eg: Vygotsky 1987), who viewed “inner speech as 
resulting from the internalisation of linguistically 
mediated interactions that are used to regulate cognition 
and behaviour” (Borghi and Fernyhough 2022 p3). 

Borghi and Fernyhough (2022) distinguished between 
different types of inner speech, including “inner 
monitoring” (eg: “Do I know this word?”) and “i 


inner 
social metacognition” (or inner dialogue) (eg: “Yes! We 
used it while talking about...”). 

Dove (2022) offered two ways in which inner speech 
helps with uncertainty: “First, it can help us monitor 
our own concepts by enabling us to trace a word’s meaning 
through its connections to other words. Rehearsing words 
in working memory can help us evaluate our understanding 
in an online fashion. Second, inner speech can engage 
discourse-related knowledge by simulating potential 
conversations concerning the concept. In other words, it 
can be used as part of an inner dialogue about the 
contents of our thoughts” (P6). 
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8. SUPERNATURAL ATTRIBUTIONS 


Exline and Wilt (2023) considered the concept of 


“supernatural attributions (SAs): explanations that 
invoke causes beyond the natural (or material, physical) 
world (p462). Supernatural beliefs and attributions are 


held by the majority of people, according to surveys (eg: 
79% of Americans in 2022 believed in God) (Exline and 
Wilt 2023). 

Supernatural beliefs include “personal agents — such 
as gods, the devil, demons, and spirits of deceased 
people — but they also include impersonal forces such as 
karma, fate, destiny, and luck. Other supernatural 
beliefs involve supernatural realms or domains. These 
might be associated with an afterlife, a domain that 
exists beyond human conceptions of time, or some form of 
ultimate reality or consciousness that extends beyond the 
material plane” (Exline and Wilt 2023 p414). “Paranormal” 
is an overlapping concept (Exline and Wilt 2023) }. 

Supernatural events are those that “suggest 
violations of natural laws or glimpses behind the veil, 
allowing people to see or experience some world beyond 
this one” (Exline and Wilt 2023 p464). These include 
miracles, near-death-experiences, and darker experiences 
related to demons. “But SAs do not require these types of 
dramatic experiences. Many people will endorse both 
natural and supernatural explanations for events... They 
may directly endorse beliefs that supernatural entities 
can work indirectly through natural events, which could 
open the door to see supernatural influence on many areas 
of life, such as health, relationships, and finances” 
(Exline and Wilt 2023 p465). The ordinariness of 
spiritual experiences is captured by the “Daily Spiritual 
Experience Scale” (Underwood 2011), for example. 

SAs can co-exist with natural or scientific 
attributions, and they are part of the culture and 
everyday lives of some people (Exline and Wilt 2023). 


The psychological aspects of SAs can be listed as 
(Exline and Wilt 2023): 


i) Coping resources in times of stress. 


ii) Causing distress - eg: temptation by evil 
spirits; bad experiences as divine punishment. 


'° “Paranormal” “typically refers to phenomena, such as psychokinesis, hauntings, and clairvoyance, 
which contradict the basic limiting principles of current scientific understanding” (Dean et al 2022 p2). 
Pechey and Halligan (2011) found that nearly four-fifths of their sample help at least one paranormal 
belief to some degree of strength. 
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iii) Guiding personal decisions - “In a national 
sample of African American adults (Debnam et al 2012), 
seeing oneself as having an active, collaborative 
partnership with God correlated with better health 
behaviours, whereas passive reliance on God correlated 
with poorer health behaviours” (Exline and Wilt 2023 
pp467-468). 


iv) Related to moral judgments - eg: adults 
identifying as transgender or gender non-conforming 
xperience more negative feelings if they believed that 
God disproved of their gender identity (Exline et al 


2021). 


v) Fuel conflict - eg: demonic attributions of 
opponents in recent US elections increased hostility 
towards them. 


Not everybody makes SAs, so what are the factors 
that predict who will and who won’t make such 
attributions? Exline and Wilt (2023) offered these 
categories of predictors: 


a) Proximal factors - attention (ie: supernatural 
ideas are accessible), reason (eg: SAs seem “logical” to 
the individual), and motivation (eg: SAS are more 
pleasant than other explanations). 


as 


b) Contextual and situational factors - Dramatic or 
anomalous experiences require “big” explanations. 
“Unusual, exciting events draw people’s attention and 
raise the ‘why?’ questions that can start an 
attributional search process. Some people, based on the 
evidence they consider, will be unable to find a natural 
explanation. They might continue to search for reasons, 
categorise the event as mysterious, or make an SA” 
(Exline and Wilt 2023 p470). 

Other situational factors include highly positive 
events, moral-related situations, being supernaturally 
tuned (eg: “spooky environment”), and life changing 
moments. 


c) Stable background factors - Four groups of 
factors can be distinguished here: 


i. Holding supernatural beliefs - “If people do not 
hold any supernatural beliefs, or if they do not believe 


in the specific entity in question, their non-belief 
should effectively close the door on SAs unless an event 
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is so compelling that it leads to a major shift in 
beliefs. But for other people, thoughts of the 
supernatural might come to mind very readily, making SAs 
easy. A major predictor here, not surprisingly, is strong 
prior belief in the entity in question” (Exline and Wilt 


2023 p472). 
ii. Psychopathology - “Many psychological disorders 
could make SAs more likely or affect their content. In 


the context of psychosis, magical thinking could cause a 
person to see illusory connections between events or 
agency behind random events; the quest for an explanation 
might lead some toward thoughts of powerful supernatural 
agents (eg: gods, the devil, fate). Peopl xperiencing 
grandiose delusions may have strong motivations to see 
themselves carrying out special missions from God — or as 
being specifically singled out for divine or karmic 
rewards. Paranoid or obsessive-compulsive tendencies 
might fuel concerns about divine punishment or demonic 
attack along with self-protective behaviours and actions 
designed to avoid these dangers. These are just a few 
possibilities” (Exline and Wilt 2023 p472). 


iii. Individual differences - eg: high need for 
closure; high absorption (ie: being deeply involved in a 
fantasy); teleological thinking (ie: the tendency to see 
events as meaningful or purposeful). Not necessarily low 


analytic thinking or high intuition as the evidence is 
divided here (appendix 8A). 


iv. Socialisation to believe in supernatural beings 
(ie: cultural differences). 


In conclusion, Exline and Wilt (2023) emphasised 
“SAs aS a normal psychological process” (p475), that are 
widespread, take many forms, and are psychological 


important for individuals. 


APPENDIX 8A - PARANORMAL BELIEFS AND COGNITIVE FUNCTION 


There is a relationship between paranormal beliefs 
and cognitive functions (eg: memory; cognitive biases; 
analytical thinking). This idea is based in part ona 
review of forty-three studies (Irwin 1993). Dean et al 
(2022) updated this review, with particular reference to 
methodological gquality. 

In May 2021 four electronic databases were searched 
using terms like “’paranormal belief’ AND thinking”. 


Psychology Miscellany No. 202; June 2024; ISSN: 1754-2200; Kevin Brewer 


Seventy-one relevant studies were found, and divided into 
Six categories - perceptual and cognitive biases; 
reasoning; intelligence, critical thinking and academic 
ability; thinking style; executive function and memory; 
and other cognition. 

Around two-thirds of studies confirmed the 
“cognitive deficit hypothesis” (ie: paranormal beliefs 
are associated with poorer cognitive performance). 
Specifically, paranormal belief was associated with an 
intuitive thinking style, confirmation bias (ie: only 
seeking evidence that supports own view), and poorer 
reasoning, though there was inconsistency in the 
findings. 

In terms of methodological issues, 60% of all 
studies had undergraduate samples, and half of these 
exclusively psychology undergraduates, which “raises 
doubt about external validity” (Dean et al 2022 pl). 

Another concern was that “studies often employ large 
numbers of measures, metrics and analyses ‘4, with no 
clearly identified primary outcome or adjustment of 
probability levels. These factors necessarily constrain 
any firm conclusions because of the high probability of 
Type 1 errors” (Dean et al 2022 p17) ?°. 

Dean et al (2022) highlighted a further concern that 
“information about non-respondents was either unreported 
or reported with insufficient detail to permit an 
assessment of potential non-response bias” (p17). 
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